使用 Python 强制另一个程序的标准输出不被缓冲
python 脚本控制 Linux 上的外部应用程序，通过管道将输入传递到外部应用程序 stdin，并通过管道从外部应用程序 stdout 读取输出。
问题在于，对管道的写入是按块而不是按行缓冲的，因此在控制脚本接收外部应用程序中的 printf 等数据输出之前会发生延迟。
无法更改外部应用程序以添加显式 fflush(0) 调用。
python标准库的pty模块如何与subprocess 模块来实现这一点？
A python script is controlling an external application on Linux, passing in input via a pipe to the external applications stdin, and reading output via a pipe from the external applications stdout.
The problem is that writes to pipes are buffered by block, and not by line, and therefore delays occur before the controlling script receives data output by, for example, printf in the external application.
The external application cannot be altered to add explicit fflush(0) calls.
How can the pty module of the python standard library be used with the subprocess module to achieve this?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
您可以使用 PTY 通过以下方式解决此问题：
You can use PTYs to solve this by:
这样做是可能的，但我能想到的唯一解决方案相当复杂、不可移植，而且可能充满了有问题的细节。您可以使用 LD_PRELOAD 使外部应用程序加载动态库，该动态库包含调用 setvbuf 来取消缓冲 stdout 的构造函数。您可能还希望将 setvbuf 包装在库中，以防止应用程序显式缓冲其自己的标准输出。并且您需要包装 fwrite 和 printf 以便它们在每次调用时刷新。编写要预加载的 .so 将使您脱离 Python。
Doing this is possible, but the only solution I can think of is fairly convoluted, non-portable, and probably fraught with problematic details. You can use LD_PRELOAD to cause the external application to load a dynamic library which contains a constructor that invokes setvbuf to unbuffer stdout. You will probably also want to wrap setvbuf in the library to prevent the application from explicitly buffering its own stdout. And you'll want to wrap fwrite and printf so that they flush on each call. Writing the .so to be preloaded will take you outside of python.
我认为这是不可能的。如果源应用程序不刷新其传出缓冲区，则数据将不会到达该进程外部，直到缓冲区溢出并强制刷新。
请注意一个完善的命令，例如 file 具有一个选项 (-n ）导致它显式刷新其输出。当在从管道读取输入文件名并打印检测到的类型的模式下使用文件时，这是必需的。由于在此模式下，文件程序在完成后不会退出，否则不会出现输出。
从较低的层面考虑这一点：输出缓冲仅仅意味着在缓冲流上执行 write() 会将数据复制到内存缓冲区中，直到缓冲区填满或（通常）直到换行被发现。然后，缓冲区中直到溢出或换行的部分被写入 write()n 到底层系统级文件描述符（可以是文件、管道、套接字等）。 ）。
我不明白你将如何说服该程序从外部刷新其缓冲区。
I don't think it's possible. If the source application doesn't flush its outgoing buffer, the data will not reach outside that process until the buffer overflows and a flush is forced.
Notice how a well-established command such as file has an option (-n) that causes it to flush its output explicitly. This is required when using file in the mode where it reads input file names from a pipe, and prints the detected type. Since in this mode, the file program doesn't quit when done, the output would otherwise not appear.
Consider this at a lower level: the output buffering simply means that doing
write()on a buffered stream copies the data into an in-memory buffer, until the buffer fills up or (typically) until a linefeed is found. Then, the part of the buffer up to the overflow or linefeed is writtenwrite()n to the underlying system-level file descriptor (which could be a file, a pipe, a socket, ...).I don't understand how you're going to convince that program to flush its buffer, from the outside.
值得注意的是，某些程序仅在认为不会发送给“真实用户”（即 tty）时才缓冲其输出。当它们检测到另一个程序正在读取它们的输出时，它们会进行缓冲。
tty 的模拟是 Expect 在自动化其他进程时所做的事情之一。
有一个 Expect 的纯 Python 实现，但我不知道它处理 tty 模拟的效果如何。
Its worth noting that some programs only buffer their output when they think it's not going to a "real user" (ie, a tty). When they detect that their output is being read by another program, they buffer.
The emulation of a tty is one of the things that Expect does in automating other processes.
There is a pure Python implementation of Expect, but I don't know how well it handles tty emulation.
这个问题的答案可能会有所帮助：
Python Run a daemon sub-process & ; read stdout
似乎正在解决同样的问题。
The answer to this question might help:
Python Run a daemon sub-process & read stdout
Seems to be addressing the same issue.
这个问题有点老了，但我认为你的问题现在可以通过使用子进程调用 stdbuf 与您要执行的命令。
This question is a bit old, but I think your problem could now be solved by using subprocess to call stdbuf with the command you want to exec.
尝试使用 -u 参数运行 Python 解释器：
这会强制 Python 使用无缓冲的 stdin/stdout，这可能会对您有所帮助。
Try running the Python interpreter with the -u argument:
This forces Python to use unbuffered stdin/stdout which may help you.