python 是否有挂钩到 EXT3

发布于 2024-07-26 04:49:37 字数 343 浏览 12 评论 0原文

我们有几个 cron 作业,通过 ftp 代理将日志记录到集中式服务器。 这些文件可能相当大并且需要一些时间来传输。 该项目的部分要求是提供一种日志记录机制,我们可以在其中记录这些传输的成功或失败。 这很简单。

我的问题是,有没有办法检查当前是否正在写入文件? 我的第一个解决方案是在给定的时间范围内检查文件大小两次并检查文件大小。 但一位同事表示,也许可以通过 python 连接到 EXT3 文件系统并检查属性以查看当前是否正在附加文件。 我的 Google-Fu 结果是空的。

是否有 EXT3 或其他模块可以让我检查文件的状态? 服务器运行带有 EXT3 文件系统的 Fedora Core 9。

We have several cron jobs that ftp proxy logs to a centralized server. These files can be rather large and take some time to transfer. Part of the requirement of this project is to provide a logging mechanism in which we log the success or failure of these transfers. This is simple enough.

My question is, is there a way to check if a file is currently being written to? My first solution was to just check the file size twice within a given timeframe and check the file size. But a co-worker said that there may be able to hook into the EXT3 file system via python and check the attributes to see if the file is currently being appended to. My Google-Fu came up empty.

Is there a module for EXT3 or something else that would allow me to check the state of a file? The server is running Fedora Core 9 with EXT3 file system.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

那请放手 2024-08-02 04:49:37

不需要 ext3 特定的钩子; 只需检查 lsof,或者更准确地说,/proc//fd/*/proc//fdinfo/* (这就是 lsof 获取其信息的地方,AFAICT)。 在那里您可以检查文件是否打开、是否可写以及“光标”位置。

这还不是全部。 但更多的工作是由 stdlib 在写入过程中在进程空间中完成的,因为大多数写入都会被缓冲,并且内核只能看到更大的数据块,因此任何“ext3 感知”监视器也不会得到这些数据。

no need for ext3-specific hooks; just check lsof, or more exactly, /proc/<pid>/fd/* and /proc/<pid>/fdinfo/* (that's where lsof gets it's info, AFAICT). There you can check if the file is open, if it's writeable, and the 'cursor' position.

That's not the whole picture; but any more is done in processpace by stdlib on the writing process, as most writes are buffered and the kernel only sees bigger chunks of data, so any 'ext3-aware' monitor wouldn't get that either.

岁月静好 2024-08-02 04:49:37

没有 ext3 挂钩可以直接检查您想要的内容。

  • 我想你可以深入研究 Fuser linux 命令的源代码,复制查找哪个进程拥有文件的部分,并监视该资源。 当没有人再打开文件时,文件传输就完成了。

另一种方法:

  • 您的 cron 作业应该告诉您它们已完成。

我们有传输文件的 cron 作业,在传输文件名后只写入一个空的filename.finished。 另一种方法是将它们转移到临时文件名,例如filename.part,然后将其重命名为filename 重命名是原子的。 在这两种情况下,您都会重复检查,直到出现 filenamefilename.finished

There's no ext3 hooks to check what you'd want directly.

  • I suppose you could dig through the source code of Fuser linux command, replicate the part that finds which process owns a file, and watch that resource. When noone longer has the file opened, it's done transferring.

Another approach:

  • Your cron jobs should tell that they're finished.

We have our cron jobs that transport files just write an empty filename.finished after it's transferred the filename. Another approach is to transfer them to a temporary filename, e.g. filename.part and then rename it to filename Renaming is atomic. In both cases you check repeatedly until the presence of filename or filename.finished

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文