分布式 Erlang 的可扩展性如何？
A 部分：
Erlang 有很多关于运行并发代理的成功案例，例如数百万个并发 Facebook 聊天。这是数百万个代理，但当然不是网络上的数百万个 CPU。当跨 LAN/WAN 进行“水平”扩展时，我无法找到有关 Erlang 扩展效果的指标。
假设我有许多（数万个）物理节点（在 Linux 上运行 Erlang），需要通过 LAN/WAN 进行通信和同步少量不频繁的数据。在什么时候我会出现通信瓶颈，不是在代理之间，而是在物理节点之间？ （或者假设网络稳定，这会起作用吗？）
B 部分：
我理解（作为 Erlang 新手，这意味着我可能完全错误）Erlang 节点尝试所有连接并意识到相互连接，形成一个 N^2 连接的点对点网络。假设 A 部分不仅仅适用于 N = 10K，Erlang 是否可以轻松配置（使用开箱即用的配置或简单的样板，而不是自己编写分组/路由算法的完整实现）以将节点集群为可管理的组并通过集群/组层次结构路由系统范围的消息？
Part A:
Erlang has a lot of success stories about running concurrent agents e.g. the millions of simultaneous Facebook chats. That's millions of agents, but of course it's not millions of CPUs across a network. I'm having trouble finding metrics on how well Erlang scales when scaling is "horizontal" across a LAN/WAN.
Let's assume that I have many (tens of thousands) physical nodes (running Erlang on Linux) that need to communicate and synchronize small infrequent amounts of data across the LAN/WAN. At what point will I have communications bottlenecks, not between agents, but between physical nodes? (Or will this just work, assuming a stable network?)
Part B:
I understand (as an Erlang newbie, meaning I could be totally wrong) that Erlang nodes attempt to all connect to and be aware of each other, resulting in an N^2 connection point-to-point network. Assuming that part A won't just work with N = 10K's, can Erlang be configured easily (using out-of-the-box config or trivial boilerplate, not writing a full implementation of grouping/routing algorithms myself) to cluster nodes into manageable groups and route system -wide messages through the cluster/group hierarchy?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我们应该明确我们谈论的是物理机器的水平可扩展性——这是唯一的问题。一台机器上的 CPU 将由一台虚拟机处理，无论虚拟机的数量是多少。
节点=机器。
首先，我可以说，您可以开箱即用（普通 OTP 安装）30-60 个节点，并在其顶部（在 Erlang 中）编写任何自定义应用程序。证明：ejabberd.
通过优化的定制应用程序，~100-150 是可能的。我的意思是，它必须是好的代码，用有关 GC、数据类型特征、消息传递等的知识编写。
超过 +150 是可以的，但是当我们谈论像 300、500 这样的数字时，它将需要优化和优化。 TCP层的定制。此外，我们的应用程序必须了解例如跨集群同步调用的成本。
另一件事是DB层。 Mnesia（内置）由于其功能在超过 20 个节点时不会有效（我的经验 - 我可能是错的）。解决方案：只需使用其他东西：Dynamo DB、单独的 MySQL 集群、HBase 等。
利用创建高质量应用程序和可扩展性的成本的最常见技术是约 20-50 个节点集群的联合。因此，在内部它是一个由约 50 个 erlang 节点组成的高效网格，并通过任何合适的协议与另外 N 个 50 个节点集群连接。综上所述，这样的系统就是N个erlang集群的联邦。
分布式 erlang 被设计为在一个数据中心运行。如果您需要更多、地理位置较远的节点，请使用联合。
有很多配置选项，例如，它们不会将所有节点相互连接。这可能会有所帮助，但是在大约 50 个集群中，erlang 开销并不大。您还可以使用“隐藏”连接创建 erlang 节点图，该连接不会加入此完整网格，但它也无法从与所有节点的连接中受益。
我发现，在这种系统中，最大的问题是将其设计为无主系统。如果你不需要那个，一切都应该没问题。
We should specify that we talk about horizontal scalability of physical machines -- that's the only problem. CPUs on one machine will be handled by one VM, no matter what the number of those is.
node = machine.
To begin, I can say that 30-60 nodes you get out of the box (vanilla OTP installation) with any custom application written on the top of that (in Erlang). Proof: ejabberd.
~100-150 is possible with optimized custom application. I means, it has to be good code, written with knowledge about GC, characteristic of data types, message passing etc.
over +150 is all right but when we talk about numbers like 300, 500 it will require optimizations & customizations of TCP layer. Also, our app has to be aware of cost of e.g. sync calls across the cluster.
The other thing is DB layer. Mnesia (built-in) due its features will not be effective over 20 nodes (my experience - I may be wrong). Solution: just use something else: dynamo DBs, separate cluster of MySQLs, HBase etc.
The most common technique to leverage cost of creating high quality application and scalability are federations of ~20-50 nodes clusters. So internally its an efficient mesh of ~50 erlang nodes and its connected via any suitable protocol with N another 50 nodes clusters. To sum up, such a system is federation of N erlang clusters.
Distributed erlang is designed to run in one data center. If you need more, geographically distant nodes, then use federations.
There are lots of config options e.g. which do not connect all nodes to each other. It may be helpful, however in ~50 cluster erlang overhead is not significant. Also you can create a graph of erlang nodes using 'hidden' connection, which doesn't join this full mesh, but also it cannot benefit from connection to all nodes.
The biggest problem I see, in this kind of systems, is designing it as master-less system. If you do not need that, everything should be ok.