如何维护长期存在的 python 项目的依赖关系和 python 版本?
简短版本:如何摆脱 python 的多版本噩梦?
长版本:多年来,我使用过Python的多个版本,更糟糕的是,Python的几个扩展(例如pygame、pylab、wxPython...)。每次它都采用不同的设置,具有不同的操作系统,有时具有不同的架构(例如我的旧 PowerPC mac)。
现在我使用的是 Mac(x86-64 上的 OSX 10.6),每次我想要恢复几个月前的脚本时,这都是一场依赖噩梦。 Python 本身已经在 /usr/bin
中提供了三种不同的风格(2.5、2.6、3.1),但我必须从 macports 为 pygame 安装 2.4,其他东西(不记得是什么)迫使我安装其他三个也都来自 macports,所以最终我很高兴拥有我系统上的 七个 (!) 个 python 实例。
但这不是问题,问题是,它们都没有安装正确的(即同一组)库,其中一些是 32 位,一些是 64 位,现在我几乎迷失了。
例如,现在我正在尝试运行一个三年前的脚本(不是我写的),它曾经使用 matplotlib/numpy 在 wxwidgets 窗口的矩形内绘制实时绘图。但我失败得很惨:来自 macports 的 py26-wxpython 无法安装,库存 python 包含 wxwidgets,但在 32 位和 64 位之间也存在一些冲突,而且它没有 numpy...真是一团糟!
显然,我做事的方式是错误的。 你通常如何应对所有这些混乱?
short version: how can I get rid of the multiple-versions-of-python nightmare ?
long version: over the years, I've used several versions of python, and what is worse, several extensions to python (e.g. pygame, pylab, wxPython...). Each time it was on a different setup, with different OSes, sometimes different architectures (like my old PowerPC mac).
Nowadays I'm using a mac (OSX 10.6 on x86-64) and it's a dependency nightmare each time I want to revive script older than a few months. Python itself already comes in three different flavours in /usr/bin
(2.5, 2.6, 3.1), but I had to install 2.4 from macports for pygame, something else (cannot remember what) forced me to install all three others from macports as well, so at the end of the day I'm the happy owner of seven (!) instances of python on my system.
But that's not the problem, the problem is, none of them has the right (i.e. same set of) libraries installed, some of them are 32bits, some 64bits, and now I'm pretty much lost.
For example right now I'm trying to run a three-year-old script (not written by me) which used to use matplotlib/numpy to draw a real-time plot within a rectangle of a wxwidgets window. But I'm failing miserably: py26-wxpython from macports won't install, stock python has wxwidgets included but also has some conflict between 32 bits and 64 bits, and it doesn't have numpy... what a mess !
Obviously, I'm doing things the wrong way. How do you usally cope with all that chaos ?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(8)
我使用 virtualenv 解决了这个问题。我同情想要避免进一步的噩梦抽象层,但 virtualenv 实际上非常干净且易于使用。您确实可以这样做(命令行,Linux):
这将创建一个新的 python 二进制文件和库位置,并默认创建到现有系统库的符号链接。然后,要切换路径以使用新环境,请执行以下操作:
就是这样。现在,如果您安装模块(例如使用
easy_install
),它们将安装到my_env
目录的lib
目录中。它们不会干扰现有的库,您不会遇到奇怪的冲突,东西不会停止在您的旧环境中工作。他们完全与世隔绝。要退出环境,只需执行以下操作:
如果您确定安装时犯了错误,或者您不再需要该环境,则只需删除该目录:
然后就完成了。真的就是这么简单。
virtualenv
很棒。 ;)I solve this using virtualenv. I sympathise with wanting to avoid further layers of nightmare abstraction, but
virtualenv
is actually amazingly clean and simple to use. You literally do this (command line, Linux):This creates a new python binary and library location, and symlinks to your existing system libraries by default. Then, to switch paths to use the new environment, you do this:
That's it. Now if you install modules (e.g. with
easy_install
), they get installed to thelib
directory of themy_env
directory. They don't interfere with existing libraries, you don't get weird conflicts, stuff doesn't stop working in your old environment. They're completely isolated.To exit the environment, just do
If you decide you made a mistake with an installation, or you don't want that environment anymore, just delete the directory:
And you're done. It's really that simple.
virtualenv
is great. ;)一些提示:
/Library/Frameworks/Python.framework
中的 python 安装。这看起来确实是一个更大的混乱,但至少事情是这样的。基本上,如果其中一个项目在 virtualenv 中运行,那么无论您执行什么升级,它都会继续运行,因为您永远不会更改“系统”安装。
Some tips:
/Library/Frameworks/Python.framework
.That seems like a bigger mess indeed, but at least things work that way. Basically, if one of the projects works in a virtualenv, it will keep working no matter what upgrades you perform, since you never change the "system" installs.
看一下 virtualenv。
Take a look at virtualenv.
我通常做的就是尝试(逐步)跟上 Python 版本的出现(并且一旦所有外部依赖项都有可用的正确版本)。
大多数时候,Python 代码本身只需进行少量修改即可按原样传输。
我最大的 Python 项目@工作(15.000+ LOC)现在已经在 Python 2.6 上运行了几个月(由于安装/检查 10+ 依赖项,从 Python 2.5 升级所有内容确实花了大半天的时间......)
总的来说,我认为这是自由软件堆栈中大多数相互依赖的组件的最佳策略(想想 Linux 软件存储库中的依赖项):保持您的版本(半)最新(或至少:以相同的速度进展)。
What I usually do is trying to (progressively) keep up with the Python versions as they come along (and once all of the external dependencies have correct versions available).
Most of the time the Python code itself can be transferred as-is with only minor needed modifications.
My biggest Python project @ work (15.000+ LOC) is now on Python 2.6 a few months (upgrading everything from Python 2.5 did take most of a day due to installing / checking 10+ dependencies...)
In general I think this is the best strategy with most of the interdependent components in the free software stack (think the dependencies in the linux software repositories): keep your versions (semi)-current (or at least: progressing at the same pace).
#!/usr/local/bin/python2.6
)我可以'不知道会出什么问题。
如果出现问题,无论如何,这可能是 macports 的错误,而不是你的错误(这是我不再使用 macports 的原因之一)。
我知道我可能遗漏了一些东西,这会被否决,但在这种情况下请至少留下一点评论,谢谢:)
#!/usr/local/bin/python2.6
)I can't see what could go wrong.
If something does, it's probably macports fault anyway, not yours (one of the reasons I don't use macports anymore).
I know I'm probably missing something and this will get downvoted, but please leave at least a little comment in that case, thanks :)
我使用 MacPorts 版本来做所有事情,但正如你所注意到的,很多默认版本都非常旧。例如,Snow Leopard 中的 vimomnicomplete 具有 python25 作为依赖项。许多与 python 相关的 port 都有旧的依赖项,但您通常可以在构建时标记较新的版本,例如
port install vim +python26
而不是port install vim +python
。在安装任何东西之前先进行一次试运行,看看是否在不需要时拉动整个 python24。经常检查 portfiles 因为 Darwin ports 的命名约定刚起步时还有一些不足之处。实际上,我只是将所有内容保留在 MacPorts 的默认/opt...
文件夹中,包括整个框架的副本以及 PyObjC 的副本等,并且一次只使用一个版本,如果出现意外情况,保留返回系统默认值的选项。为了避免使用 virtualenv,这可能有点太多的工作,我一直想抽出时间来使用它。I use the MacPorts version for everything, but as you note a lot of the default versions are bizarrely old. For example vim omnicomplete in Snow Leopard has python25 as a dependency. A lot of python related ports have old dependencies but you can usually flag the newer version at build time, for example
port install vim +python26
instead ofport install vim +python
. Do a dry run before installing anything to see if you are pulling, for example, the whole of python24 when it isn't necessary. Check portfiles often because the naming convention as Darwin ports was getting off the ground left something to be desired. In practice I just leave everything in the default/opt...
folders of MacPorts, including a copy of the entire framework with duplicates of PyObjC, etc., and just stick with one version at a time, retaining the option to return to the system default if things break unexpectedly. Which is all perhaps a bit too much work to avoid usingvirtualenv
, which I've been meaning to get around to using.我使用 Buildout 很幸运。您可以设置一个列表,其中包含您想要的鸡蛋和版本。然后,Buildout 会为您下载并安装每个版本的私有版本。它创建了一个私有的“python”二进制文件,并且已经安装了所有的eggs。本地“nosetests”使调试变得容易。您可以使用自己的功能来扩展构建。
不利的一面是,Buildout 可能相当神秘。执行“buildout -vvvv”一段时间,看看它到底在做什么以及为什么。
http://www.buildout.org/docs/tutorial.html
I've had good luck using Buildout. You set up a list of which eggs and which versions you want. Buildout then downloads and installs private versions of each for you. It makes a private "python" binary with all the eggs already installed. A local "nosetests" makes things easy to debug. You can extend the build with your own functions.
On the down side, Buildout can be quite mysterious. Do "buildout -vvvv" for a while to see exactly what it's doing and why.
http://www.buildout.org/docs/tutorial.html
至少在Linux下,多个Python可以相当愉快地共存。我在 CentOS 系统上使用 Python 2.6,该系统需要 Python 2.4 作为各种系统的默认设置。我只是将 python 2.6 编译并安装到一个单独的目录树中(并将适当的 bin 目录添加到我的路径中),这相当轻松。然后通过输入“python2.6”来调用它。
一旦你启动并运行了单独的 python,安装特定版本的库就很简单了。如果您使用所需的 python 调用 setup.py 脚本,它将安装在适合该 python 的目录中,并且脚本将安装在与 python 可执行文件本身相同的目录中,并且在调用时将自动使用正确的 python。
我也尽量避免使用太多的库。当我只需要库(例如 scipy)中的一两个函数时,我经常会看看是否可以将它们复制到我自己的项目中。
At least under Linux, multiple pythons can co-exist fairly happily. I use Python 2.6 on a CentOS system that needs Python 2.4 to be the default for various system things. I simply compiled and installed python 2.6 into a separate directory tree (and added the appropriate bin directory to my path) which was fairly painless. It's then invoked by typing "python2.6".
Once you have separate pythons up and running, installing libraries for a specific version is straightforward. If you invoke the setup.py script with the python you want, it will be installed in directories appropriate to that python, and scripts will be installed in the same directory as the python executable itself and will automagically use the correct python when invoked.
I also try to avoid using too many libraries. When I only need one or two functions from a library (eg scipy), I'll often see if I can just copy them to my own project.