Python检查文件状态是否正在上传

发布于 2024-09-24 18:51:56 字数 216 浏览 3 评论 0原文

Python 2.6

我的脚本需要监视ftp上的一些1G文件,当它被更改/修改时,脚本会将其下载到另一个地方。这些文件名将保持不变,人们会先删除ftp上的原始文件,然后上传更新的版本。我的脚本将检查文件元数据,例如文件大小和修改日期,以查看是否有任何差异。

问题是当脚本检查元数据时,新文件可能仍在上传。遇到这种情况该如何处理呢?是否有任何文件属性指示上传状态(例如文件被锁定)?谢谢。

Python 2.6

My script needs to monitor some 1G files on the ftp, when ever it's changed/modified, the script will download it to another place. Those file name will remain unchanged, people will delete the original file on ftp first, then upload a newer version. My script will checking the file metadata like file size and date modified to see if any difference.

The question is when the script checking metadata, the new file may be still being uploading. How to handle this situation? Is there any file attribute indicates uploading status (like the file is locked)? Thanks.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

花开半夏魅人心 2024-10-01 18:51:56

没有这样的属性。您可能无法获取此类文件,但这取决于服务器软件。此外,文件访问标志可以在文件上传时以一种方式设置,然后在上传完成时更改;或者不完整的文件可能已修改名称(例如original_filename.ext.part)——这完全取决于用于上传的服务器端软件。

如果您控制服务器,请创建自己的元数据,例如,上传完成后,在新上传的文件旁边创建一个空标志文件。

在一般情况下,恐怕您能做的最好的事情就是监视文件大小,如果文件大小一段时间内没有变化,则认为文件已完全上传。使该间隔足够大(大约分钟)。

There is no such attribute. You may be unable to GET such file, but it depends on the server software. Also, file access flags may be set one way while the file is being uploaded and then changed when upload is complete; or incomplete file may have modified name (e.g. original_filename.ext.part) -- it all depends on the server-side software used for upload.

If you control the server, make your own metadata, e.g. create an empty flag file alongside the newly uploaded file when upload is finished.

In the general case, I'm afraid, the best you can do is monitor file size and consider the file completely uploaded if its size is not changing for a while. Make this interval sufficiently large (on the order of minutes).

最后的乘客 2024-10-01 18:51:56

你的问题遗漏了一些细节,但我会尽力回答。

  • 如果您正在运行状态检查器
    同一服务器上的程序
    运行 ftp:

1) 根据您的操作系统,如果您使用的是 Linux 并且已将 inotify 内置到内核中,则可以使用 pyinotify 来监视您的上传目录 - inotify 与打开、修改、关闭事件区分开来,并允许您异步监视文件系统事件,这样您就不必不断轮询。 OSX 和 Windows 都有类似但实现方式不同的功能。

2)你可以用Python方式 tail -f< /a> 查看新文件何时放置在服务器上(如果您正在记录该文件),并在看到相关更新消息时进行更新。

  • 如果您远程运行程序

3) 如果您的状态检查实用程序必须在 FTP 服务器的远程主机上运行,​​则必须轮询文件状态并构建一些逻辑来检测大小更改。您可以使用 FTP 'SIZE' 命令来获得易于解析的字符串。

您必须在其中添加一些逻辑,这样如果文件大小变小,您就会认为它正在被替换,然后等待它变大,直到它停止增长并在一段时间内保持相同的大小。如果存档以可以验证总和的方式压缩,则可以下载它、校验和,然后重新上传到远程站点。

Your question leaves out a few details, but I'll try to answer.

  • If you're running your status checker
    program on the same server thats
    running ftp:

1) Depending on your operating system, if you're using Linux and you've built inotify into your kernel you could use pyinotify to watch your upload directory -- inotify distinguishes from open, modify, close events and lets you asynchronously watch filesystem events so you're not polling constantly. OSX and Windows both have similar but differently implemented facilities.

2) You could pythonically tail -f to see when a new file is put on the server (if you're even logging that) and just update when you see related update messages.

  • If you're running your program remotely

3) If your status checking utility has to run on a remote host from the FTP server, you'd have to poll the file for status and build in some logic to detect size changes. You can use the FTP 'SIZE' command for this for an easily parse-able string.

You'd have to put some logic into it such that if the filesize gets smaller you would assume it's being replaced, and then wait for it to get bigger until it stops growing and stays the same size for some duration. If the archive is compressed in a way that you could verify the sum you could then download it, checksum, and then reupload to the remote site.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文