使用 async_read_until 时 boost::asio 是否截断输出?

发布于 2024-10-31 00:52:19 字数 2707 浏览 6 评论 0原文

这个把我难住了

如果我使用 async_read_until 调用,打印超过 4k 字节的内容似乎会出现问题?

我有一个小函数可以打印 100 行(刚刚超过 4k)。

除非我注册 async_read_until 回调,否则在每种组合中都可以正常工作。那时我的输出被截断为大约 4k。请注意,并非总是如此,有时会更少,有时会打印整个内容,这似乎与机器上的负载有关,几乎就像发生了一些超时?一些 asio 线程的东西?无论如何,如果我注释掉 async_read_until 调用,无论我调用 printLines 多少次,它每次都能正常工作。我什至可以使用 ioService post 功能,它工作正常......

发生了什么事?顺便说一句,我使用的是linux和amd64机器gcc4.4。 (Redhat)


使用 linux 'strace' 我得到了更多线索:

似乎在使用 fcntl 调用调用 async_read_until、asio 之后,导致我的输出文件描述符改变行为?
一段时间后它退出打印输出:

select(4, [0 3], [], [], {300, 0}) = 1(在 [0] 中,左 {298, 830000})
readv(0, [{"\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ 0\0\0\0\0\0\0\0\0\0\0"..., 512}], 1) = 1
write(2, "这是一个长句子的测试"..., 93) = 93
write(2, "这是一个长句子的测试"..., 93) = 93
...大约40次
write(2, "这是一个长句子的测试"..., 93) = 53
write(2, "这是一个长句子的测试"..., 93) = -1 EAGAIN (资源暂时不可用)
write(2, "这是一个长句子的测试"..., 93) = -1 EAGAIN (资源暂时不可用)
write(2, "这是一个长句子的测试"..., 93) = -1 EAGAIN (资源暂时不可用)
write(2, "这是一个长句子的测试"..., 93) = -1 EAGAIN (资源暂时不可用)
write(2, "这是一个长句子的测试"..., 93) = -1 EAGAIN (资源暂时不可用)
...剩下的直到达到 100。

所以你可以看到选择循环等待我输入返回键。然后我们称我为空 读取处理程序并退出 ioService。此时我调用 printLines 函数并尝试打印 100 行,但在打印 40 行后它退出了。

此 EAGAIN 的某些原因导致输出停止写入。
再次,如果我不调用 async_read_until 我的 printf 不会损坏。


我想我知道发生了什么,似乎 Asio 正在将我的标准输出文件描述符转换为异步非阻塞模式我请求标准输入文件描述符上的异步读取模式。这就是为什么在一些输出之后我在写入时收到 EAGAIN 错误。当然 printf 会忽略这些,因此我的输出被截断。不知道这是 Asio 中的错误还是 Linux 上的副作用?


这是我复制问题的简单程序:

#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <iostream>

using namespace boost::asio;
using namespace std;

io_service ioService;
boost::asio::streambuf inStream;
posix::stream_descriptor input(ioService, STDIN_FILENO);

void printLines()
{
    for (int i = 0; i < 100; i++) {
        fprintf(stderr, "This is a test of a long sentence, there will be %d more sentences after this on is printed.\n", i);
    }
    fflush(stderr);
}

void readHandler(const boost::system::error_code& error)
{ // Don't care about read!! }

int main()
{
    boost::system::error_code ec;
    //printLines(); << this works if uncommented
    //ioService.post(printLines); << this works if uncommented

    boost::asio::async_read_until(input, inStream, "\n",
    bind(readHandler, placeholders::error)); // causes truncated output

    cout << "Hit Return to continue..." << endl;
    ioService.run_one( ec );
    assert(!ec);
    printLines(); // partial output if async_read_until is called?
    return 0;
}

This one has me stumped!

There seems to be a problem printing out anything over 4k bytes if I use the async_read_until call?

I have a little function that prints out 100 lines (just over 4k).

Works fine in every combination except if I register a async_read_until callback. At that point my output is truncates to about 4k. Note not always though, sometimes less and sometimes the whole thing is printed it seems to be related to the load on the machine, almost like there is some timeout going on? Some asio threading thing? Anyway if I comment out the async_read_until call it works fine every time no matter how many time I call printLines. I can even use the ioService post function it works fine...

What is going on? BTW, I'm using linux and amd64 machine gcc4.4. (Redhat)


Using linux 'strace' I got some more clues:

It seems like after calling async_read_until, asio using fcntl calls, causes my output file descriptor to change behaviour?
it quits printing output after awhile:

select(4, [0 3], [], [], {300, 0}) = 1 (in [0], left {298, 830000})
readv(0, [{"\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512}], 1) = 1
write(2, "This is a test of a long sentenc"..., 93) = 93
write(2, "This is a test of a long sentenc"..., 93) = 93
... about 40 times
write(2, "This is a test of a long sentenc"..., 93) = 53
write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "This is a test of a long sentenc"..., 93) = -1 EAGAIN (Resource temporarily unavailable)
... for the rest until 100 is reached.

So you can see the select loop waiting for my input return key. Then we call my empty
read handler and drop out of the ioService. At this time I call my printLines function and try to print 100 lines but it quits after printing 40 some.

Something with this EAGAIN is causing the output to stop writting.
Again if I don't call async_read_until my printf does not get corrupted.


I think I know what is going on, seems like Asio is turning my stdout file desciptor into async non-blocking mode when I request async read mode on the stdin file descriptor. That's why after some output I get EAGAIN errors on writes. Of course printf ignores those so my output is truncated. Don't know if that is a bug in Asio or just a side effect on linux?


Here is my simple program to duplicate the problem:

#include <boost/asio.hpp>
#include <boost/bind.hpp>
#include <iostream>

using namespace boost::asio;
using namespace std;

io_service ioService;
boost::asio::streambuf inStream;
posix::stream_descriptor input(ioService, STDIN_FILENO);

void printLines()
{
    for (int i = 0; i < 100; i++) {
        fprintf(stderr, "This is a test of a long sentence, there will be %d more sentences after this on is printed.\n", i);
    }
    fflush(stderr);
}

void readHandler(const boost::system::error_code& error)
{ // Don't care about read!! }

int main()
{
    boost::system::error_code ec;
    //printLines(); << this works if uncommented
    //ioService.post(printLines); << this works if uncommented

    boost::asio::async_read_until(input, inStream, "\n",
    bind(readHandler, placeholders::error)); // causes truncated output

    cout << "Hit Return to continue..." << endl;
    ioService.run_one( ec );
    assert(!ec);
    printLines(); // partial output if async_read_until is called?
    return 0;
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

青朷 2024-11-07 00:52:19

如果我在 printLines 之前插入对 ioctl 的以下调用,

int opt = 0;
ioctl( STDIN_FILENO, FIONBIO, &opt );
printLines();

则该行为是预期的

samm@macmini ~> ./a.out
Hit Return to continue...

This is a test of a long sentence, there will be 0 more sentences after this on is printed.
...
This is a test of a long sentence, there will be 96 more sentences after this on is printed.
This is a test of a long sentence, there will be 97 more sentences after this on is printed.
This is a test of a long sentence, there will be 98 more sentences after this on is printed.
This is a test of a long sentence, there will be 99 more sentences after this on is printed.
samm@macmini ~>

虽然我不清楚为什么会发生这种情况,但这可能是 ioctl 的副作用,这主要是特定于内核/设备的。在浏览各个手册页时,我没有看到任何描述这种行为的内容。您可以尝试从在 printLines 中使用 fprintf 切换到使用 posix::stream_descriptorSTDERR_FILENO


编辑:看起来 Boost.Asio 发生了一些最近的更改解决这种行为。具体来说

  • 添加了新的 non_blocking() 函数
    管理非阻塞行为
    套接字或描述符。这
    io_control() 命令名为
    non_blocking_io 现已弃用
    喜欢这些新功能。

  • 添加了新的native_non_blocking()
    管理功能
    底层的非阻塞模式
    套接字或描述符。这些功能
    旨在允许
    任意封装
    非阻塞系统调用如
    异步操作,以这样的方式
    对用户来说是透明的
    套接字对象。函数没有
    对行为的影响
    套接字的同步操作
    或描述符。

我看起来很有趣。我在 Mac 上使用 Boost 1.45 来运行您的重现器,这些更改将出现在即将发布的 Boost 1.47 中。我将尝试获取 Chris 最新的 asio 开发版本,看看行为是否发生了变化。

If I insert the following call to ioctl prior to printLines

int opt = 0;
ioctl( STDIN_FILENO, FIONBIO, &opt );
printLines();

the behavior is expected

samm@macmini ~> ./a.out
Hit Return to continue...

This is a test of a long sentence, there will be 0 more sentences after this on is printed.
...
This is a test of a long sentence, there will be 96 more sentences after this on is printed.
This is a test of a long sentence, there will be 97 more sentences after this on is printed.
This is a test of a long sentence, there will be 98 more sentences after this on is printed.
This is a test of a long sentence, there will be 99 more sentences after this on is printed.
samm@macmini ~>

Though it's not clear to me why this happens, it could be a side effect of ioctl, which is largely kernel/device specific. I did not see anything describing this behavior when trolling around the various man pages. You might try switching from using fprintf in printLines to using a posix::stream_descriptor with STDERR_FILENO instead.


Edit: it looks like there have been some recent changes to Boost.Asio that may address this behavior. Specifically

  • Added new non_blocking() functions for
    managing the non-blocking behaviour of
    a socket or descriptor. The
    io_control() commands named
    non_blocking_io are now deprecated in
    favour of these new functions.

  • Added new native_non_blocking()
    functions for managing the
    non-blocking mode of the underlying
    socket or descriptor. These functions
    are intended to allow the
    encapsulation of arbitrary
    non-blocking system calls as
    asynchronous operations, in a way that
    is transparent to the user of the
    socket object. The functions have no
    effect on the behaviour of the
    synchronous operations of the socket
    or descriptor.

look interesting to me. I used Boost 1.45 on my mac to run your reproducer, and these changes will be in the upcoming Boost 1.47. I'll try to grab Chris's latest asio development release and see if the behavior has changed.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文