如何避免 Python 文件输入缓冲
可能的重复:
为 sys.stdin 设置较小的缓冲区大小?
我有一个 Python ( 2.4/2.7) 使用 fileinput
从标准输入或文件读取的脚本。它很容易使用,并且除了一种情况外运行良好:
tail -f log | filter.py
问题是我的脚本缓冲了它的输入,而(至少在这种情况下)我想立即看到它的输出。这似乎源于以下事实:fileinput 在执行任何操作之前使用 readlines() 来获取其 bufsize 字节值。我尝试使用 1 的 bufsize ,但它似乎没有帮助(这有点令人惊讶)。
我确实发现我可以编写这样不缓冲的代码:
while 1:
line = sys.stdin.readline()
if not line: break
sys.stdout.write(line)
这样做的问题是我失去了文件输入功能(即它自动打开传递给我的程序的所有文件,或者如果没有则打开标准输入,并且它甚至可以自动解压缩输入文件)。
那么我怎样才能两全其美呢?理想情况下,我不需要显式管理我的输入文件列表(包括解压缩),但以“流”方式使用时不会延迟输入。
Possible Duplicate:
Setting smaller buffer size for sys.stdin?
I have a Python (2.4/2.7) script using fileinput
to read from standard input or from files. It's easy to use, and works well except for one case:
tail -f log | filter.py
The problem is that my script buffers its input, whereas (at least in this case) I want to see its output right away. This seems to stem from the fact that fileinput uses readlines()
to grab up to its bufsize
worth of bytes before it does anything. I tried using a bufsize of 1 and it didn't seem to help (which was somewhat surprising).
I did find that I can write code like this which does not buffer:
while 1:
line = sys.stdin.readline()
if not line: break
sys.stdout.write(line)
The problem with doing it this way is that I lose the fileinput functionality (namely that it automatically opens all the files passed to my program, or stdin if none, and it can even decompress input files automatically).
So how can I have the best of both? Ideally something where I don't need to explicitly manage my input file list (including decompression), and yet which doesn't delay input when used in a "streaming" way.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
尝试运行 python -u ; man 说它将“强制 stdin、stdout 和 stderr 完全无缓冲”。
您只需更改
filter.py
第一行的 hashbang 路径即可。Try running
python -u
; man says that it will "force stdin, stdout and stderr to be totally unbuffered".You can just alter the hashbang path at the first line of
filter.py
.您尝试过吗:
没有测试过,但是通过阅读 openhook 参数的作用以及通过 0 来打开 bufsize 参数,这应该可以解决问题。
Have you tried:
Not tested it, but from reading what openhook param does and what passing 0 to open for bufsize param, this should do the trick.