python中有工作线程之类的东西吗?
我编写了一个漂亮的多线程脚本,当我运行它时,它在 25 个线程下的性能比直接调用线程处理程序的性能更差。
然后我发现了全局解释器锁。我想问,在我放弃这个脚本的Python并用其他东西重写它之前,有没有什么方法可以在Python中进行实际工作的多线程?
I wrote a beautiful multithreaded script, and when I ran it, it performed worse with 25 threads than with just direct invocation of the thread handler.
Then I discovered the global interpreter lock. I want to ask, before I discard python for this script and rewrite the thing in something else, is there any way to do actual working multithreading in python?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
另一种方法是放弃线程,而是使用 Multiprocessing 模块(Python 2.6+),它绕过了 GIL,并且有一个至少与线程模块中的 API 相似的 API。
The other approach is to abandon threads and instead use the Multiprocessing module (Python 2.6+), which gets around the GIL, and has an API that's at least similar to the one in the threading module.
现在这是一个有趣的问题 - 我认为在直接的 CPython 中没有转义 GIL 的方法。
Stackless Python 应该通过使用 微线程,但我不认为它逃脱了 GIL。
此外,根据 python.org 上的 GIL 页面,Jython 和 IronPython 没有 GIL 。
Now this is an interesting question - I don't think there's an escaping the GIL in straight CPython.
Stackless Python is supposed to have improved "concurrency" performance over CPython with its use of microthreads, but I dont't think it escapes the GIL.
Additionally, according to the GIL page on python.org, Jython and IronPython don't have a GIL.
正确答案很大程度上取决于您正在做什么。
大量 CPU 密集型(和阻塞 IO 密集型)任务(例如压缩和图像渲染)通常使用本机代码完成,本机库通常会在工作时释放 GIL,从而允许并发。当您可以将 CPU 密集型工作隔离到狭窄的本机调用时,您将获得并发性、重要的本机性能以及使用 Python 编写大部分代码的便利性。
并非所有代码都具有可以在本机库中整齐实现的小型、可隔离的计算代码块,但很多代码都具有。
The correct answer depends heavily on what you're doing.
Heavily CPU-bound (and blocking IO-bound) tasks, like compression and image rendering, are usually done with native code, and native libraries normally release the GIL while they work, which allows concurrency. When you can isolate the CPU-intensive work to a narrow native call, you get concurrency, native performance where it counts, and the convenience of writing most of the code in Python.
Not all code has those small, isolatable blocks of computational code that can be neatly implemented in a native library, but a whole lot do.
据我所知,在 CPython(Python 的 C 实现)中,没有。 (如果我错了,请纠正我,我很想知道是否存在解决方案!)。
您可能对 IronPython (.NET) 或 JPython (Java/JWM) 感兴趣。我还没有使用过它们,但我相信至少其中之一支持给定执行环境的本机线程。
In CPython (the implementation of Python in C), no, according to my knowledge. (If I'm wrong, please correct me, I'd sure like to know if a solution exists!).
You might want to be interested in IronPython (.NET) or JPython (Java/JWM). I haven't used them but I believe at least one of them supports threads native to the given execution environment.
如果适用于您的问题,您可以尝试 multiprocessing 模块。
You could try the multiprocessing module, if it's applicable to your problem.