使用 python 记录文件和进程超时的输出
我正在浏览一大堆文件并对所有文件运行命令。我想记录输出,如果该命令对一个文件花费的时间超过 5 分钟,我想停止该命令并转到下一个文件。
我有两个问题:
我想将文件名记录到输出文件,并记录输出消息。我使用
Popen
来记录消息并使用communicate
来记录消息,但我使用write()
写入的所有文件名都没有直到整个任务完成后才开始写作。我不知道如何轮询该进程并在 5 分钟后退出并转到下一个文件。
下面是简化的代码:
import os, fnmatch
import subprocess
import sys
f=open('filenames','w')
'Locate all files matching supplied filename pattern in and below supplied root directory.'''
def locate(pattern, root=os.curdir):
for path, dirs, files in os.walk(os.path.abspath(root)):
for filename in fnmatch.filter(files, pattern):
yield os.path.join(path, filename)
for filename in locate("*.dll"):
f.write(filename)
#cmd defintion is changed for simplicity
cmd='cat %s' %filename
p=subprocess.Popen(cmd,stdout=f)
p.communicate()[0]
I am going through a big list of files and running a command on all of them. I want to log the output and if the command takes more than 5 minutes for a file, I would like to stop the command and go to the next file.
I have two problems:
I want to record the file name to the output file and also record the output messages. I am using
Popen
to log the messages and usingcommunicate
so it gets logged, but all the filenames I am writing withwrite()
don't get written till the whole task is done.I am not sure how I can poll the process and quit it after 5 minutes and go to the next file.
Below is simplified code:
import os, fnmatch
import subprocess
import sys
f=open('filenames','w')
'Locate all files matching supplied filename pattern in and below supplied root directory.'''
def locate(pattern, root=os.curdir):
for path, dirs, files in os.walk(os.path.abspath(root)):
for filename in fnmatch.filter(files, pattern):
yield os.path.join(path, filename)
for filename in locate("*.dll"):
f.write(filename)
#cmd defintion is changed for simplicity
cmd='cat %s' %filename
p=subprocess.Popen(cmd,stdout=f)
p.communicate()[0]
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
f.write(filename)
之后使用f.flush()
。f.flush()
afterf.write(filename)
.我大致使用的是:
基本上:在单独的线程内启动进程。这是必要的,因为
threading
模块会为您提供超时等待,而subprocess
模块则不会。等待超时;检查线程是否还活着。如果线程处于活动状态,我们会通过终止进程来使其终止(这里,subprocess
为您提供一个kill()
构造,其中threading
没有)并等待线程自行结束。请记住,communicate() 会阻止并存储子级 stderr 和 stdout 的完整输出,因此如果输出非常大,这可能不起作用。
Here's what I use, roughly:
Basically: start the process inside a separate thread. This is necessary because the
threading
module gives you timeout waiting, while thesubprocess
module does not. Wait until the timeout; check if the thread is still alive. If the thread is live, we cause it to terminate by killing the process (here,subprocess
gives you akill()
construct, whichthreading
does not) and waiting for the thread to end on its own accord.Keep in mind that
communicate()
blocks and stores the full output from the child's stderr and stdout, so this may not work if the output is very large.