如何让 Ruby 或 Python 网站使用多核?
即使 Python 和 Ruby 每个解释器线程有一个内核线程,它们有一个全局解释器锁(GIL),用于保护潜在共享的数据结构,因此这会抑制多处理器执行。 尽管这些语言中用 C 或 C++ 编写的部分可以是自由线程的,但对于纯解释代码来说这是不可能的,除非您使用多个进程。 实现这一目标的最佳方法是什么? 使用 FastCGI? 创建虚拟化服务器集群或群? 使用它们的 Java 等效项 JRuby 和 Jython?
Even though Python and Ruby have one kernel thread per interpreter thread, they have a global interpreter lock (GIL) that is used to protect potentially shared data structures, so this inhibits multi-processor execution. Even though the portions in those languajes that are written in C or C++ can be free-threaded, that's not possible with pure interpreted code unless you use multiple processes. What's the best way to achieve this? Using FastCGI? Creating a cluster or a farm of virtualized servers? Using their Java equivalents, JRuby and Jython?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
使用在单独的解释器中运行每个响应的接口,例如用于 Python 的
mod_wsgi
。 这使得可以在不遇到 GIL 的情况下使用多线程。编辑:显然,mod_wsgi不再支持每个进程多个解释器,因为白痴无法弄清楚如何正确实现扩展模块。 不过,它仍然支持在 FastCGI 风格的单独进程中运行请求,因此这显然是当前可接受的解决方案。
Use an interface that runs each response in a separate interpreter, such as
mod_wsgi
for Python. This lets multi-threading be used without encountering the GIL.EDIT: Apparently,
mod_wsgi
no longer supports multiple interpreters per process because idiots couldn't figure out how to properly implement extension modules. It still supports running requests in separate processes FastCGI-style, though, so that's apparently the current accepted solution.在 Python 和 Ruby 中,只能使用多个核心,以便产生新的(重量级)进程。
Java 对应项继承了 Java 平台的可能性。 您可能暗示使用 Java 线程。 例如,这就是为什么有时(通常)将 Java 应用程序服务器(如 Glassfish)用于 Ruby on Rails 应用程序的原因。
In Python and Ruby it is only possible to use multiple cores, is to spawn new (heavyweight) processes.
The Java counterparts inherit the possibilities of the Java platform. You could imply use Java threads. That is for example a reason why sometimes (often) Java Application Server like Glassfish are used for Ruby on Rails applications.
对于 Python,PyProcessing 项目允许您使用进程进行编程,就像使用线程一样。 它作为
multiprocessing
包含在最近发布的 2.6 版本的标准库中。 该模块具有许多用于建立和控制对共享数据结构(队列、管道等)的访问的功能,并支持常见的习惯用法(即管理器和工作池)。For Python, the PyProcessing project allows you to program with processes much like you would use threads. It is included in the standard library of the recently released 2.6 version as
multiprocessing
. The module has many features for establishing and controlling access to shared data structures (queues, pipes, etc.) and support for common idioms (i.e. managers and worker pools).使用 Rails 执行此操作的“标准”方法是运行 Mongrel 实例的“包”(即:rails 应用程序的 4 个副本),然后使用 apache 或 nginx 或其他一些软件坐在它们前面并执行操作作为负载平衡器。
这可能是其他 ruby 框架(例如 merb 等)的实现方式,但我个人没有使用过这些框架。
操作系统将负责在其自己的 CPU 上运行每个杂种。
如果您安装 mod_rails aka phusion guest 它也会为您启动和停止 Rails 进程的多个副本,因此它最终会以类似的方式将负载分散到多个 CPU/核心上。
The 'standard' way to do this with rails is to run a "pack" of Mongrel instances (ie: 4 copies of the rails application) and then use apache or nginx or some other piece of software to sit in front of them and act as a load balancer.
This is probably how it's done with other ruby frameworks such as merb etc, but I haven't used those personally.
The OS will take care of running each mongrel on it's own CPU.
If you install mod_rails aka phusion passenger it will start and stop multiple copies of the rails process for you as well, so it will end up spreading the load across multiple CPUs/cores in a similar way.
我不完全确定您想要解决哪个问题,但是如果您使用 mod_python 通过 apache prefork MPM 部署 python/django 应用程序,apache 将启动多个工作进程来处理不同的请求。
如果一个请求需要如此多的资源,以至于您想使用多个核心,请查看 pyprocessing。 但我认为这并不明智。
I'm not totally sure which problem you want so solve, but if you deploy your python/django application via an apache prefork MPM using mod_python apache will start several worker processes for handling different requests.
If one request needs so much resources, that you want to use multiple cores have a look at pyprocessing. But I don't think that would be wise.