将 GILed Python 程序限制在单个 CPU 上会提高性能吗?

发布于 2024-11-19 19:30:57 字数 233 浏览 2 评论 0原文

David Beazley 的关于 Python 和 GIL 的论文之后,限制Python 程序(带 GIL 的 CPython 等)到基于 Windows 的多核系统中的单个 CPU?

它会提高性能吗?

更新:假设使用多个线程(不确定是否有区别)

Following up on David Beazley's paper regarding Python and GIL, would it be a good practice to limit a Python program (CPython with GIL and all) to a single CPU in a Windows based multi-core system?

Would it boost performance?

UPDATE: Assume multiple threads are used (not sure if it makes a difference)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

谁与争疯 2024-11-26 19:30:57

该论文确实暗示,将程序限制为单核可以提高特定情况下的性能。但是,您需要处理许多问题:

  1. 他的测试主要针对计算密集型线程,而不是 IO 绑定线程。如果您使用的线程经常自动阻塞(例如在等待客户端的 Web 服务器中),那么您根本不会遇到 GIL 问题。
  2. GIL 问题专门处理线程,而不是进程。我可能读错了你的问题,但你似乎是在问将所有 Python 程序限制为单个核心。使用进程进行并行的程序不会遇到 GIL 问题,并且将它们限制为单个核心会使它们变慢。
  3. GIL 在 Python 3.2 中截然不同(正如 David 在 此视频。GIL 已被明确更改以处理此类问题。虽然仍然存在问题,但它没有 。

时,您才会希望通过强制操作系统将程序限制为单核来使您的生活变得复杂

  1. 多线程
  2. 计算密集型
  3. 总而言之,只有当您运行低于 Python 3.2 的

多核机器上的程序。

The paper does indeed imply that limiting a program to a single core would improve performance in that particular case. However, there are a number of concerns that you would need to deal with:

  1. His tests are mainly for compute intensive threads rather than IO bound threads. If the threads you are using often block voluntarily (such as in a web server waiting for a client) then you don't run into GIL issues at all.
  2. The GIL issues deal specifically with threads and not processes. I may be reading your question wrong, but you seem to be asking about restricting all Python programs to a single core. Programs using processes for parallelism don't suffer from GIL issues and restricting them to a single core will make them slower.
  3. The GIL is drastically different in Python 3.2 (as David mentions in this video. The GIL was changed explicitly to deal with such issues. While it still has problems, it no longer has this problem.

In summary, the only time you want to complicate your life by forcing the OS to restrict the program to a single core is when you are running a:

  1. Multithreaded
  2. Compute Intensive
  3. Lower than Python 3.2

program on a multicore machine.

简单 2024-11-26 19:30:57

偏差:对于涉及繁重 CPU 处理的并行计算,我非常喜欢
与线程编程相比,更喜欢消息传递和协作进程
(当然,这取决于问题)

你不应该将你的程序限制在一个核心上。 Beazley 只是演示了一个特定问题,该问题在这些独特条件下(这些条件是 IO 绑定线程和 CPU 绑定线程相互竞争)表现不佳。理想情况下,您希望通过使用不同的方法(导入多处理)来避免这些情况。

我认为最好的解决方案是使用多处理模块将 CPU 绑定任务放入其他进程中,以便它们利用自己的内核,并将 IO 绑定任务放入线程(或微线程/协程,如果您阅读了他的有趣论文:http://www.dabeaz.com/coroutines/),因为 GIL 最适合这些类型的任务。

结论:Python 线程最适合 IO 密集型任务,而非 CPU 密集型任务

Bias : For parallel computing involving heavy CPU processing, I much
prefer message passing and cooperating processes to thread programming
(of course, it depends on the problem)

You shouldn't limit your programs to one core. Beazley was just demonstrating a specific problem that performed poorly under those unque conditions (those conditions being an IO bound thread and a CPU bound thread competing against each other). Ideally you want to avoid those conditions by using a different method (import multiprocessing).

I think the best solution is to put your CPU bound tasks in other processes using the multiprocessing module so that they utilize their own cores, and IO bound tasks in threads (or microthreads/coroutines, if you read his interesting paper on that: http://www.dabeaz.com/coroutines/) since the GIL is best suited for those types of tasks.

Conclusion: Python threads are best suited for IO bound tasks, NOT CPU bound.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文