Erlang 进程如何(如果有的话)映射到内核线程?
Erlang 因能够支持许多轻量级进程而闻名; 它可以做到这一点,因为这些不是传统意义上的进程,甚至不是像 P 线程那样的线程,而是完全在用户空间中的线程。
这很好(实际上很棒)。 那么 Erlang 线程如何在多核/多处理器环境中并行执行呢? 当然,它们必须以某种方式映射到内核线程才能在单独的内核上执行?
假设情况确实如此,那么这是如何做到的呢? 许多轻量级进程是否映射到单个内核线程?
或者还有其他方法可以解决这个问题吗?
Erlang is known for being able to support MANY lightweight processes; it can do this because these are not processes in the traditional sense, or even threads like in P-threads, but threads entirely in user space.
This is well and good (fantastic actually). But how then are Erlang threads executed in parallel in a multicore/multiprocessor environment? Surely they have to somehow be mapped to kernel threads in order to be executed on separate cores?
Assuming that that's the case, how is this done? Are many lightweight processes mapped to a single kernel thread?
Or is there another way around this problem?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
答案取决于所使用的 VM:
1) 非 SMP:有一个调度程序(操作系统线程),它执行所有 Erlang 进程,取自可运行进程池(即那些没有被
接收
阻塞的进程)2) SMP:有K个调度程序(操作系统线程,K通常是CPU核心的数量),它从共享进程队列执行Erlang进程。 它是一个简单的 FIFO 队列(带有锁以允许多个操作系统线程同时访问)。
3) R13B 及更新版本中的 SMP:将会有 K 个调度程序(和以前一样)从多个进程队列执行 Erlang 进程。 每个调度程序都有自己的队列,因此将添加从一个调度程序到另一个调度程序的进程迁移逻辑。 该解决方案将通过避免共享进程队列中的过度锁定来提高性能。
有关详细信息,请参阅由爱立信 AB 的 Kenneth Lundin 为 Erlang 用户准备的本文档会议,斯德哥尔摩,2008 年 11 月 13 日。
Answer depends on the VM which is used:
1) non-SMP: There is one scheduler (OS thread), which executes all Erlang processes, taken from the pool of runnable processes (i.e. those who are not blocked by e.g.
receive
)2) SMP: There are K schedulers (OS threads, K is usually a number of CPU cores), which executes Erlang processes from the shared process queue. It is a simple FIFO queue (with locks to allow simultaneous access from multiple OS threads).
3) SMP in R13B and newer: There will be K schedulers (as before) which executes Erlang processes from multiple process queues. Each scheduler has it's own queue, so process migration logic from one scheduler to another will be added. This solution will improve performance by avoiding excessive locking in shared process queue.
For more information see this document prepared by Kenneth Lundin, Ericsson AB, for Erlang User Conference, Stockholm, November 13, 2008.
我想修改一下之前的答案。
Erlang,或者更确切地说,Erlang 运行时系统 (erts),将调度程序(操作系统线程)的数量和运行队列的数量默认为平台上处理元素的数量。 那是处理器核心或硬件线程。 您可以使用以下方法在运行时更改这些设置:
Erlang 进程尚未与任何调度程序有任何关联。 平衡调度程序之间的进程的逻辑遵循两个规则。 1)饥饿的调度程序会窃取另一个调度程序的工作。 2) 迁移路径设置为将进程从具有大量进程的调度程序推送到工作量较少的调度程序。 这样做是为了确保每个进程减少计数(执行时间)的公平性。
然而,调度程序可以被锁定到特定的处理元素。 默认情况下不这样做。 要让 erts 执行调度程序->核心关联性,请使用:
可以在文档中找到其他几种绑定类型。 使用affinity可以大大提高重负载情况下的性能! 特别是在锁争用较多的情况下。 而且,Linux 内核至少不能处理超线程。 如果你的平台上有超线程,你应该在 erlang 中使用这个功能。
I want to ammend previous answers.
Erlang, or rather the Erlang runtime system (erts), defaults the number of schedulers (OS threads) and the number of runqueues to number of processing elements on your platform. That is processors cores or hardware threads. You can change these settings in runtime using:
The Erlang processes does not have any affinity to any schedulers yet. The logic balancing the processes between the schedulers follows two rules. 1) A starving scheduler will steal work from another scheduler. 2) Migration paths are setup to push processes from schedulers with lots of processes to schedulers with less work. This is done to assure fairness in reduction count (execution time) for each process.
Schedulers however can be locked to specific processing elements. This not done by default. To let erts do the scheduler->core affinity use:
Several other bind types can be found in the documentation. Using affinity can greatly improve performance in heavy load situations! Especially in high lock contention situations. Also, the linux kernel cannot handle hyperthreads to say the least. If you have hyperthreads on your platform you should really use this feature in erlang.
我在这里纯粹是猜测,但我想象有少量线程,它们从公共进程池中选择进程来执行。 一旦进程遇到阻塞操作,执行它的线程就会将其放在一边并选择另一个。 当正在执行的进程导致另一个进程解除阻塞时,新解除阻塞的进程将被放入池中。 我想线程也可能会停止进程的执行,即使它没有在某些点被阻止以服务其他进程。
I'm purely guessing here, but I'd imagine that there's a small number of threads, which pick processes from a common process pool for execution. Once a process hits a blocking operation, the thread executing it puts it aside and picks another. When a process being executed causes another process to become unblocked, that newly unblocked process gets placed into the pool. I suppose a thread might also stop execution of a process even when it's not blocked at certain points to serve other processes.
我想在已接受的答案中添加一些内容。
Erlang Scheduler 是 Erlang 运行时系统的重要组成部分,并提供了自己对操作系统线程之上的轻量级进程概念的抽象和实现。
每个调度程序在单个操作系统线程中运行。 通常,硬件上有多少个 CPU(核心)就有多少个调度程序(尽管它是可配置的,但当调度程序的数量超过硬件核心的数量时,自然不会带来太大的价值)。 系统还可以配置为调度程序不会在操作系统线程之间跳转。
现在,当Erlang进程被创建时,ERTS和调度程序完全负责管理生命周期和资源消耗以及内存占用等。
核心实现细节之一是每个进程的时间预算为2000当调度程序从运行队列中选取该进程时,可以进行减少。 系统中的每个进度(甚至 I/O)都保证有一个缩减预算。 这实际上使 ERTS 成为具有抢占式多任务处理功能的系统。
我会推荐 Jesper Louis Andersen 撰写的关于该主题的精彩博客文章 http ://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html
简而言之:Erlang 进程不是操作系统线程,不会直接映射到它们。 Erlang 调度程序在操作系统线程上运行,并提供更细粒度的 Erlang 进程的智能实现,将这些细节隐藏在程序员的眼睛后面。
I would like to add some input to what was described in the accepted answer.
Erlang Scheduler is the essential part of the Erlang Runtime System and provides its own abstraction and implementation of the conception of lightweight processes atop the OS threads.
Each Scheduler runs within a single OS thread. Normally, there are as many schedulers as CPU (cores) are on he hardware (it is configurable though and naturally does not bring much value when number of schedulers exceeds those of hardware cores). The system might also be configured that scheduler will not jump between OS threads.
Now, when the Erlang process is being created it is entirely the responsibility of the ERTS and Scheduler to manage life cycle and resources consumption as well as its memory footprint etc.
One of the core implementation details is that each process has a time budget of 2000 reductions available when the Scheduler picks up that process from the run queue. Each progress in the system (even I/O) is guaranteed to have a reductions budget. That is what actually makes ERTS a system with preemptive multitasking.
I would recommend a great blog post on that topic by Jesper Louis Andersen http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html
As the short answer: Erlang processes are not OS threads and do not map to them directly. Erlang Schedulers are what runs on the OS threads and provide smart implementation of more finely grained Erlang processes hiding those details behind programmer's eyes.