加快 Python 命令行应用程序的初始执行速度

发布于 2024-12-10 08:53:52 字数 3791 浏览 0 评论 0原文

我使用 Python 和 argparse。现在它基本上只是启动 Vim,然后将缓冲区推入 SQLite 数据库。

问题是,在我的两台机器(~2/3GHz Intel Core 2 Duos)上加载 Python 都很慢,而且最基本的功能(在启动时打印帮助菜单)可能需要一秒钟以上的时间。我知道我的代码很好,因为 Python 速度非常快,并且一旦 Python 加载,交互模式就很快,但我可以用一个简单的 Hello Word 来模仿我的烦恼:

$ time python -c "print 'hello world'"
hello world
real    0m0.669s
user    0m0.070s
sys     0m0.041s

当然,这个问题并不是 Python 独有的:

$ time erl -noshell -eval 'io:fwrite("Hello, World!\n"), init:stop().'
Hello, World!
real    0m2.824s
user    0m0.253s
sys     0m0.104s

我的问题是:如何加快 Python 应用程序的初始执行速度?我希望我的程序感觉像 gitwc

系统:我在 OS X 10.6.8 上使用 python2.6 时遇到此问题,在 OS X 10.7.2 上使用 python2.7 时也遇到此问题。

注意:python(和 erl)的后续执行要快得多,但我已经在对该程序进行测试,并且希望它真正快速。

更新:我尝试运行pypy,发现它的初始加载时间与python2相似。我的两个系统上的 python2.6 和 2.7(初始加载时约为 0.5 秒),与 OS X 10.6.8 上的 python2.6 相比,后续调用的性能只有一半(pypy 约为 0.08 秒,2.6 约为 0.35 秒),并且与 OS X 10.7.2 上的 python2.7 相比,后续调用的性能相似(pypy python2.7 为 ~.08s)。

更新 2: jimbob 博士建议的输出(这似乎将初始加载时间缩短了 2/3)-

$ python -vSEc "print 'hello world'"
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
import encodings # directory /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.py
import encodings # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.pyc
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.py
import codecs # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.pyc
import _codecs # builtin
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.py
import encodings.aliases # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.pyc
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.py
import encodings.utf_8 # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.pyc
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
hello world
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] zipimport
# cleanup[1] _codecs
# cleanup[1] signal
# cleanup[1] encodings
# cleanup[1] encodings.utf_8
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _warnings
# cleanup[1] codecs
# cleanup sys
# cleanup __builtin__
# cleanup ints: 3 unfreed ints
# cleanup floats

real    0m0.267s
user    0m0.009s
sys     0m0.043s

I've prototyped a command line application for quick note-taking using Python and argparse. Right now it basically just launches Vim and then shoves the buffer into a SQLite database.

The problem is, loading Python is slow on both of my machines (~2/3GHz Intel Core 2 Duos) and the most basic functionality (printing a help menu on startup) can take over a second. I know my code is fine because Python is perfectly fast and the interactive mode is snappy once Python loads, but I can mimic my annoyance with a simple Hello Word:

$ time python -c "print 'hello world'"
hello world
real    0m0.669s
user    0m0.070s
sys     0m0.041s

Of course, the problem is not unique to Python:

$ time erl -noshell -eval 'io:fwrite("Hello, World!\n"), init:stop().'
Hello, World!
real    0m2.824s
user    0m0.253s
sys     0m0.104s

My question is: How can I speed up the initial execution of a Python application? I want my program to feel like git or wc.

System: I encounter this problem with python2.6 on OS X 10.6.8 and with python2.7 on OS X 10.7.2.

Note: Subsequent executions of python (and erl) are much faster, but I'm already dogfooding this program and I want it to be truly snappy.

Update: I've tried running pypy and find it has similar initial load time to python2.6 and 2.7 on both of my systems (~.5 seconds on initial load), half the performance on subsequent calls compared to python2.6 on OS X 10.6.8 (~.08s for pypy, ~.35s for 2.6), and similar performance on subsequent calls compared to python2.7 on OS X 10.7.2 (~.08s for pypy and python2.7).

Update 2: Output from dr jimbob's suggestion (this seems to trim the initial load time by 2/3) -

$ python -vSEc "print 'hello world'"
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
import encodings # directory /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.py
import encodings # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/__init__.pyc
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.py
import codecs # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/codecs.pyc
import _codecs # builtin
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.py
import encodings.aliases # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/aliases.pyc
# /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.pyc matches /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.py
import encodings.utf_8 # precompiled from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/encodings/utf_8.pyc
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
hello world
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] zipimport
# cleanup[1] _codecs
# cleanup[1] signal
# cleanup[1] encodings
# cleanup[1] encodings.utf_8
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _warnings
# cleanup[1] codecs
# cleanup sys
# cleanup __builtin__
# cleanup ints: 3 unfreed ints
# cleanup floats

real    0m0.267s
user    0m0.009s
sys     0m0.043s

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

你怎么这么可爱啊 2024-12-17 08:53:52

您可以利用臭名昭著的“Microsoft Office Quick Start”或“Java Quick Start”以及 IIRC 甚至“Adobe Fast-something”所使用的类似技术...

诀窍是尝试并保留所有的库程序始终在磁盘缓存中。

您可以使用简单的 crontab 命令来获取它,该命令被编程为每小时运行一次。确切的细节将取决于您的系统,但应该使用类似的东西:

$ crontab -e
0 * * * * python -c 'pass'

虽然对我来说更有效,但您应该编写一个简单的Python脚本来导入您的程序正在使用的所有模块,然后结束。

You can make use of a similar technique used by the infamous "Microsoft Office Quick Start" or the "Java Quick Start" and, IIRC even the "Adobe Fast-something"...

The trick is to try and keep all the libraries of the program in the disk cache, all the time.

You can get it with a simple crontab command, programmed to be run once each hour. The exact details will depend on your system, but should work with something like:

$ crontab -e
0 * * * * python -c 'pass'

Although to me more effective you should write a simple Python script that imports all the modules your program is using and then just ends.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文