维护 subprocess.Popen 命令之间的环境状态？

发布于 2024-10-08 11:54:35 字数 1591 浏览 4 评论 0原文

我正在为我们的系统编写一个部署引擎，其中每个项目都指定他的自定义部署指令。

节点在 EC2 上运行。

其中一个项目依赖于第三方应用程序的源版本。

具体来说：

cd /tmp
wget s3://.../tools/x264_20_12_2010.zip
unzip x264_20_12_2010.zip
cd x264_20_12_2010
./configure
make
checkinstall --pkgname=x264 --pkgversion "2:0.HEAD" --backup=no --deldoc=yes --fstrans=no --default

目前我正在使用 boto 的 ShellCommand （内部使用 subprocess.Popen）执行此操作，这看起来像这样：

def deploy():
        ShellCommand("apt-get remove ffmpeg x264 libx264-dev")
        ShellCommand("apt-get update")
        ShellCommand("apt-get install -y build-essential checkinstall yasm texi2html libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support libfaac-dev libjack-jackd2-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl1.2-dev libtheora-dev libvorbis-dev libvpx-dev libx11-dev libxfixes-dev libxvidcore-dev zlib1g-dev")

        ShellCommand("cd /tmp")
        s3cmd_sync("s3://.../tools/x264_20_12_2010.zip", "/tmp/x264_20_12_2010.zip")
        ShellCommand("unzip x264_20_12_2010.zip")
        ShellCommand("cd x264_20_12_2010")
        ShellCommand("./configure")
        ShellCommand("make")
        ShellCommand(r'checkinstall --pkgname=x264 --pkgversion "2:0.HEAD" --backup=no --deldoc=yes --fstrans=no --default')

遗憾的是，这失败了，因为 cd /tmp 适用于子进程，这意味着一旦我返回父进程并发出第二个 ShellCommand exeenv 是从父进程继承的，这让我认为我需要一些 shell 命令的执行框架，它将在同一个子进程中应用所有命令，而不会丢失上下文。

这个问题的推荐解决方案是什么？请注意，命令行执行的应用程序的日志记录非常重要（没有它如何进行调试？），这就是我喜欢 ShellCommand 的原因...（如果感兴趣，请参阅 boto 日志记录）。

谢谢你，
格言。

原文

I'm writing a deployment engine for our system, where each project specifies his custom deployment instructions.

The nodes are running on EC2.

One of the projects depends on a from source version of a 3rd party application.

Specifically:

cd /tmp
wget s3://.../tools/x264_20_12_2010.zip
unzip x264_20_12_2010.zip
cd x264_20_12_2010
./configure
make
checkinstall --pkgname=x264 --pkgversion "2:0.HEAD" --backup=no --deldoc=yes --fstrans=no --default

Currently I'm doing this with boto's ShellCommand (which uses subprocess.Popen internally), this looks something like this:

def deploy():
        ShellCommand("apt-get remove ffmpeg x264 libx264-dev")
        ShellCommand("apt-get update")
        ShellCommand("apt-get install -y build-essential checkinstall yasm texi2html libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support libfaac-dev libjack-jackd2-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl1.2-dev libtheora-dev libvorbis-dev libvpx-dev libx11-dev libxfixes-dev libxvidcore-dev zlib1g-dev")

        ShellCommand("cd /tmp")
        s3cmd_sync("s3://.../tools/x264_20_12_2010.zip", "/tmp/x264_20_12_2010.zip")
        ShellCommand("unzip x264_20_12_2010.zip")
        ShellCommand("cd x264_20_12_2010")
        ShellCommand("./configure")
        ShellCommand("make")
        ShellCommand(r'checkinstall --pkgname=x264 --pkgversion "2:0.HEAD" --backup=no --deldoc=yes --fstrans=no --default')

Sadly this fails, because cd /tmp applies to the subprocess, meaning that once I return the the father process and issue the second ShellCommand the exeenv is inherited from the father, which leans me to think that I need some execution framework for shell commands which will apply all commands in the same sub process without loosing context.

What is the recommend solution to this problem? Please note that loggings of the command line executed app is very important (how can you debug without it?), which is why I like ShellCommand... (see boto logging if interested).

Thank you,
Maxim.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

潦草背影 2024-10-15 11:54:35

想想 os.chdir("DIRECTORY") 而不是 Popen("cd DIRECTORY")
也许是最好不要为每个命令执行一个新的 shell：只需编写一个多行 shell skript

deploy_commands = "" “apt-get foo
apt-get 酒吧
cd巴兹; boo bat"""

end 通过 Popen(deploy_commands, shell=True) 执行。

但是请阅读 Popen 文档中有关不转义不受信任参数的安全警告。

回复收藏 0 原文

你如我软肋 2024-10-15 11:54:35

我最终做了这个

def shell_script(appname, *commands):
        workspace = tempfile.mkdtemp(prefix=appname + '-')
        installer = open(workspace + "/installer.sh", 'w')

        installer.write("#!/bin/bash\n")
        installer.write("cd " + workspace + "\n")
        for line in commands:
                installer.write(line + "\n")
        ShellCommand("chmod u+x " + installer.name)
        installer.close()
        ShellCommand(installer.name)

I ended up doing this

def shell_script(appname, *commands):
        workspace = tempfile.mkdtemp(prefix=appname + '-')
        installer = open(workspace + "/installer.sh", 'w')

        installer.write("#!/bin/bash\n")
        installer.write("cd " + workspace + "\n")
        for line in commands:
                installer.write(line + "\n")
        ShellCommand("chmod u+x " + installer.name)
        installer.close()
        ShellCommand(installer.name)

回复收藏 0 原文

~没有更多了~