将 GILed Python 程序限制在单个 CPU 上会提高性能吗?
继 David Beazley 的关于 Python 和 GIL 的论文之后,限制Python 程序(带 GIL 的 CPython 等)到基于 Windows 的多核系统中的单个 CPU?
它会提高性能吗?
更新:假设使用多个线程(不确定是否有区别)
Following up on David Beazley's paper regarding Python and GIL, would it be a good practice to limit a Python program (CPython with GIL and all) to a single CPU in a Windows based multi-core system?
Would it boost performance?
UPDATE: Assume multiple threads are used (not sure if it makes a difference)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
该论文确实暗示,将程序限制为单核可以提高特定情况下的性能。但是,您需要处理许多问题:
时,您才会希望通过强制操作系统将程序限制为单核来使您的生活变得复杂
多核机器上的程序。
The paper does indeed imply that limiting a program to a single core would improve performance in that particular case. However, there are a number of concerns that you would need to deal with:
In summary, the only time you want to complicate your life by forcing the OS to restrict the program to a single core is when you are running a:
program on a multicore machine.
你不应该将你的程序限制在一个核心上。 Beazley 只是演示了一个特定问题,该问题在这些独特条件下(这些条件是 IO 绑定线程和 CPU 绑定线程相互竞争)表现不佳。理想情况下,您希望通过使用不同的方法(
导入多处理
)来避免这些情况。我认为最好的解决方案是使用多处理模块将 CPU 绑定任务放入其他进程中,以便它们利用自己的内核,并将 IO 绑定任务放入线程(或微线程/协程,如果您阅读了他的有趣论文:http://www.dabeaz.com/coroutines/),因为 GIL 最适合这些类型的任务。
结论:Python 线程最适合 IO 密集型任务,而非 CPU 密集型任务。
You shouldn't limit your programs to one core. Beazley was just demonstrating a specific problem that performed poorly under those unque conditions (those conditions being an IO bound thread and a CPU bound thread competing against each other). Ideally you want to avoid those conditions by using a different method (
import multiprocessing
).I think the best solution is to put your CPU bound tasks in other processes using the multiprocessing module so that they utilize their own cores, and IO bound tasks in threads (or microthreads/coroutines, if you read his interesting paper on that: http://www.dabeaz.com/coroutines/) since the GIL is best suited for those types of tasks.
Conclusion: Python threads are best suited for IO bound tasks, NOT CPU bound.