如何运行 python for-loop 在后台使用 nohup 执行终端命令?
我目前正在使用 os.system 在 python 脚本中运行 bash 命令,如下所示:
for bucket in bucket_lst:
start = time.time()
command = "gsutil rsync -r /home/imagenet/tf_records " + bucket
os.system(command)
end = time.time() - start
time_lst.append(end)
我在这里所做的是将数据从 Google 计算引擎传输到不同区域的 Google Cloud Storage,这些区域存储在“bucket_lst, ”并测量完成到每个区域的传输所需的时间。
每次传输到一个区域大约需要一到两个小时,并且大约有 30 个区域,因此我需要使用 nohup 在后台运行此过程,因为与 GCE 的 ssh 连接经常断开。
目前,我尝试了命令“nohup python3 gce_to_gcs_throuhgput.py”,但似乎它在运行 for 循环执行的命令的第一次迭代后结束了该进程。为什么会发生这种情况?如何解决问题,以便 nohup 命令运行,直到将数据传输到每个区域?
I am currently running bash commands in python script using os.system as the following:
for bucket in bucket_lst:
start = time.time()
command = "gsutil rsync -r /home/imagenet/tf_records " + bucket
os.system(command)
end = time.time() - start
time_lst.append(end)
What I'm doing here is to transfer the data from a Google Compute Engine to Google Cloud Storage in diverse regions, which the regions are stored in "bucket_lst," and measure the time taken to finish the transfer to each region.
Each transfer to a region takes about an hour to two hours, and there are about 30 regions, so I need to run this process in the background with nohup as the ssh connection to the GCE gets disconnected often.
Currently, I tried the command "nohup python3 gce_to_gcs_throuhgput.py", but it seems like it ends the process after running the very first iteration of the command executed by the for-loop. Why is this happening and how can I fix things so the nohup command runs until it transfers the data to every regions?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
将
os.system()
替换为subprocess.run()
按 @gonzalo-odiard 提供的方式工作。您可以按照 Python 文档 的子流程部分替换它。要了解两者的区别,请检查此SO post。
Replacing
os.system()
tosubprocess.run()
worked as provided by @gonzalo-odiard. And you can replace it by following the subprocess section of the Python docs.To know the difference of the two, check this SO post.