如何在 Perl 中检测目录大小的变化
我试图找到一种在 Perl 中监视目录的方法,特别是目录的大小,并在检测到目录大小的变化时执行特定的操作。
我遇到的问题是大文件需要花费大量时间才能复制到此目录中,即> 100MB。发生的情况(在 Windows 中,而不是 Unix)是系统为整个文件保留足够的磁盘空间,即使文件仍在复制过程中。这会给我带来问题,因为我的脚本将尝试对该尚未完成复制的文件执行操作。我可以通过 'du' 轻松检测 Unix 中的目录大小变化,但 Windows 中的 'du' 的行为方式不同。
Perl 中有没有准确的方法来检测目录大小的变化?
编辑:需要澄清的一些要点: - 我的 Perl 脚本仅监视特定目录,并在检测到新文件或新目录时,对此新文件或目录执行操作。它不复制任何文件;网络上的用户会将文件复制到我正在监视的目录中。 - 当出现(已复制,未移动)的新文件或目录非常大(> 100MB,但通常为几 GB)并且我的程序在此复制完成之前触发时,就会出现问题 - 在 Unix 中,我可以轻松地“du”看到有问题的文件/目录的大小正在增长,并采取适当的操作 - 在 Windows 中,大小是静态的,因此我无法检测到此更改 - opendir/readdir/closedir 不可行,因为出现的一些目录可能包含数千个文件,我想避免
理想情况下我希望我的程序在更改时触发的开销,但我不知道如何做这个。截至目前,它正忙着等待,直到检测到变化。文件/目录大小的变化不在我的控制范围内。
I am trying to find a way of monitoring directories in Perl, in particular the size of a directory, and upon detecting a change in directory size, perform a particular action.
The issue I have is with large files that require a noticeable amount of time to copy into this directory, i.e. > 100MB. What happens (in Windows, not Unix) is the system reserves enough disk space for the entire file, even though the file is still copying in progress. This causes problems for me, because my script will try to perform an action on this file that has not finished copying over. I can easily detect directory size changes in Unix via 'du', but 'du' in Windows does not behave the same way.
Are there any accurate methods of detecting directory size changes in Perl?
Edit: Some points to clarify:
- My Perl script is only monitoring a particular directory, and upon detecting a new file or a new directory, perform an action on this new file or directory. It is not copying any files; users on the network will be copying files into the directory I am monitoring.
- The problem occurs when a new file or directory appears (copied, not moved) that is significantly large (> 100MB, but usually a couple GB) and my program fires before this copy completes
- In Unix I can easily 'du' to see that the file/directory in question is growing in size, and take the appropriate action
- In Windows the size is static, so I cannot detect this change
- opendir/readdir/closedir is not feasible, as some of the directories that appear may contain thousands of files, and I want to avoid the overhead of
Ideally I would like my program to be triggered on change, but I am not sure how to do this. As of right now it busy waits until it detects a change. The change in file/directory size is not in my control.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
您似乎正在解决根本问题,而不是解决它 - 您的程序在完成复制文件时未正确发送通知。为什么不这样做而不是使用特定于操作系统的机制来尝试间接确定操作何时完成?
You seem to be working around the underlying issue rather than addressing it -- your program is not properly sending a notification when it is finished copying a file. Why not do that instead of using OS-specific mechanisms to try to indirectly determine when the operation is complete?
您可以使用 Linux::Inotify2 或 Win32::ChangeNotify 检测目录/文件更改。
编辑: File::ChangeNotify 似乎是一个更好的选择(跨平台&由 < a href="http://www.catalystframework.org/" rel="nofollow noreferrer">催化剂)
You can use Linux::Inotify2 or Win32::ChangeNotify to detect directory/file changes.
EDIT: File::ChangeNotify seems a better option (cross-platform & used by Catalyst)
据我了解,您正在轮询包含数千个文件的目录。当您看到新文件时,系统会对该文件执行操作。如果文件正在使用或仍在复制,这会导致问题,对吗?
可能有几种解决方案:
1) 使用 flock 检测文件是否仍在被另一个进程使用(测试它是否在您的操作系统、文件系统和 Perl 版本上正常工作)。
2) 在 Windows 上使用 LockFile 调用。如果失败,则操作系统或其他进程正在使用该文件。
3) 将轮询间隔更改为服务器上的非繁忙时间,并在进程完成时使目录脱机。
As I understand it, you are polling a directory with thousands of files. When you see a new file, there is an action that is taken on the file. This causes problems if the file is in use or still being copied, correct?
There are potentially several solutions:
1) Use flock to detect if the file is still in use by another process (test if it works properly on your OS, file system, and Perl version).
2) Use a LockFile call on Windows. If it fails, the OS or another process is using that file.
3) Change the poll interval to a non busy time on the server and take the directory off line while your process completes.
除了最缺乏经验的 Perl 程序员之外,所有人都应该能够评估目录的大小。如果您了解以下内容,则可以用 15 行代码编写您自己的可移植版本的
du
:glob
或opendir / readdir / Closedir
进行迭代通过目录中的文件-f file
、-d file
等)来区分常规文件和目录名称stat 函数或文件大小运算符
-s file
获取文件的大小Evaluating the size of a directory is something all but the most inexperienced Perl programmers should be able to do. You can write your own portable version of
du
in 15 lines of code if you know about:glob
oropendir / readdir / closedir
to iterate through the files in a directory-f file
,-d file
, etc.) to distinguish between regular files and directory namesstat
function or file size operator-s file
to obtain the size of a file有一个很好的模块,称为 File::Monitor,它将检测新文件、已删除文件、大小变化以及可以使用 stat 完成的任何其他属性。然后它会为您输出文件。
http://metacpan.org/pod/File::Monitor
您设置了基线扫描,然后为您要查找的每个项目设置回调,这样您就可以通过以下方式看到新的更改:
如果您需要比一级更深,只需执行到您需要的任何级别即可。完成此操作后,它会找到新文件,您可以触发应用程序对文件执行您想要的操作。
There is a nice module called File::Monitor, it will detect new files, deleted files, changes in size and any other attribute that can be done with stat. It will then go and out put the files for you.
http://metacpan.org/pod/File::Monitor
You set up a baseline scan, then set up a call back for each item you are looking for, so new changes you can see via
If you need to go deeper than one level just do it to whatever level you need. After this is done and it finds new files you can trigger you application to do what you want on the files.