有效读取大型、活跃的 Python 日志文件
当我的 Python 脚本使用 Python 内置日志库逐行将大量日志写入文本文件时,在我的 Delphi 支持的 Windows 程序中,我希望有效读取所有新添加的日志(行)。
当 Python 脚本记录日志时 到该文件,我的 Windows 程序将 保留一个只读文件句柄 该日志文件;
我将使用 Windows API 来获取 日志文件时通知 改变了;文件更改后,它将读取新附加的行。
我是 Python 新手,您认为这种方法有任何可能的问题吗? Python 日志库是否锁定整个日志?谢谢!
When my Python script is writing a large amount of logs to a text file line by line using the Python built-in logging library, in my Delphi-powered Windows program I want to effectively read all newly added logs (lines).
When the Python scripting is logging
to the file, my Windows program will
keep a readonly file handle to
that log file;I'll use the Windows API to get
informed when the log file is
changed; Once the file is changed, it'll read the newly appended lines.
I'm new to Python, do you see any possible problem with this approach? Does the Python logging lib lock the entire log? Thanks!
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
当然,这取决于您使用的日志记录处理程序,但正如您从 源代码,logging.FileHandler当前不创建任何文件锁。默认情况下,它以“a”(追加)模式打开文件,因此只要您的 Windows 调用可以处理该问题,就应该没问题。
It depends on the logging handler you use, of course, but as you can see from the source code, logging.FileHandler does not currently create any file locks. By default, it opens files in 'a' (append) mode, so as long as your Windows calls can handle that, you should be fine.
正如 ʇs֙ɹoɈ 评论的那样,标准 FileHandler 记录器不会锁定文件,因此它应该可以工作。但是,如果由于某种原因您无法锁定文件 - 那么我建议您的其他应用程序定期打开该文件,记录其读取的位置,然后稍后再返回到该点。我知道 Linux DenyHosts 程序在处理必须长时间监视的日志文件时使用这种方法。在这些情况下,简单地持有锁是不可行的,因为目录可能会移动,文件会被轮换等等。尽管它确实使事情变得复杂,但你必须在某个地方以持久状态存储文件名+读取位置。
As ʇsәɹoɈ commented, the standard FileHandler logger does not lock the file, so it should work. However, if for some reason you cannot keep you lock on the file - then I'd recommend having your other app open the file periodically, record the position it's read to and then seek back to that point later. I know the Linux DenyHosts program uses this approach when dealing with log files that it has to monitor for a long period of time. In those situations, simply holding a lock isn't feasible, since directories may move, the file get rotated out, etc. Though it does complicate things in that then you have to store filename + read position in persistent state somewhere.