在 Erlang 集群中的所有节点上运行 gen_server 的最佳方法是什么？
我正在 Erlang 中构建一个监控工具。当在集群上运行时，它应该在所有节点上运行一组数据收集功能，并在单个“记录器”节点上使用 RRD 记录该数据。
当前版本有一个在主节点 (rolf_node_sup) 上运行的 Supervisor，它尝试在集群中的每个节点 (rolf_service_sup) 上运行第二个 Supervisor。然后，每个节点上的管理程序应该启动并监视一组进程，这些进程将消息发送回主节点上的 gen_server (rolf_recorder)。
这仅适用于本地。任何远程节点上都没有启动管理程序。我使用以下代码尝试加载来自记录器节点的节点主管：
rpc:call(Node, supervisor, start_child, [{global, rolf_node_sup}, [Services]])
我发现有几个人建议主管实际上只为本地进程设计。例如
什么是实现我的要求（在集群中的所有节点上运行受监督代码）的最 OTP 方法吗？
- 建议使用分布式应用程序作为分布式管理树的替代方案。这些不适合我的用例。它们提供节点之间的故障转移，但保持代码在一组节点上运行。
- pool 模块很有趣。但是，它允许在当前负载最少的节点上运行作业，而不是在所有节点上运行。
- 或者，我可以在主服务器上创建一组受监督的“代理”进程（每个节点一个），使用 proc_lib:spawn_link 在每个节点上启动一个监督程序。如果节点上出现问题，代理进程应该终止，然后由其主管重新启动，而主管又应该重新启动远程进程。 slave 模块在这里可能非常有用。
- 或者也许我过于复杂化了。直接监督节点是一个坏主意，也许我应该构建应用程序以更松散耦合的方式收集数据。通过在多个节点上运行应用程序来构建集群，告诉一个节点成为主节点，就这样吧！
一些要求：
- 架构应该能够应对节点加入和离开池而无需人工干预。
- 为了简单起见，我想构建一个单主解决方案，至少在最初是这样。
- 在我的实现中，我更愿意使用现有的 OTP 工具，而不是手动编写的代码。
I'm building a monitoring tool in Erlang. When run on a cluster, it should run a set of data collection functions on all nodes and record that data using RRD on a single "recorder" node.
The current version has a supervisor running on the master node (rolf_node_sup) which attempts to run a 2nd supervisor on each node in the cluster (rolf_service_sup). Each of the on-node supervisors should then start and monitor a bunch of processes which send messages back to a gen_server on the master node (rolf_recorder).
This only works locally. No supervisor is started on any remote node. I use the following code to attempt to load the on-node supervisor from the recorder node:
rpc:call(Node, supervisor, start_child, [{global, rolf_node_sup}, [Services]])
I've found a couple of people suggesting that supervisors are really only designed for local processes. E.g.
What is the most OTP way to implement my requirement to have supervised code running on all nodes in a cluster?
- A distributed application is suggested as one alternative to a distributed supervisor tree. These don't fit my use case. They provide for failover between nodes, but keeping code running on a set of nodes.
- The pool module is interesting. However, it provides for running a job on the node which is currently the least loaded, rather than on all nodes.
- Alternatively, I could create a set of supervised "proxy" processes (one per node) on the master which use
proc_lib:spawn_linkto start a supervisor on each node. If something goes wrong on a node, the proxy process should die and then be restarted by it's supervisor, which in turn should restart the remote processes. The slave module could be very useful here. - Or maybe I'm overcomplicating. Is directly supervising nodes a bad idea, instead perhaps I should architect the application to gather data in a more loosely coupled way. Build a cluster by running the app on multiple nodes, tell one to be master, leave it at that!
Some requirements:
- The architecture should be able to cope with nodes joining and leaving the pool without manual intervention.
- I'd like to build a single-master solution, at least initially, for the sake of simplicity.
- I would prefer to use existing OTP facilities over hand-rolled code in my implementation.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
有趣的挑战，有多种解决方案。以下只是我的建议，希望能让您更好地选择如何编写程序。
据我了解您的程序，您希望有一个主节点来启动您的应用程序。这将在集群中的节点上启动 Erlang VM。
pool模块使用slave模块来执行此操作，这需要双向基于密钥的 ssh 通信。它还要求您有正确的 DNS 工作。slave的一个缺点是，如果 master 死亡，slave 也会随之死亡。这是设计使然，因为它可能完美地适合原始用例，但是在您的情况下，它可能很愚蠢（例如，即使主服务器已关闭，您可能仍然希望收集数据）至于 OTP 应用程序，每个节点都可能运行相同的应用程序。在代码中，您可以使用配置或发现来确定集群中的节点角色。
我建议使用一些操作系统工具或守护程序工具或类似工具来启动 Erlang VM。每个虚拟机都将启动相同的应用程序，其中一个将作为主机启动，其余的作为从机启动。这样做的缺点是，在集群中的机器上“自动”运行软件变得更加困难，就像使用
slave所做的那样，但它也更加健壮。在每个应用程序中，您都可以根据节点的角色拥有合适的监督树。消除节点间的监督和生成使系统变得更加简单。
我还建议将所有节点推送到主节点。这样，主节点实际上不需要关心从节点中发生的情况，它甚至可能忽略节点已关闭的事实。这也允许添加新节点而不需要对主节点进行任何更改。 cookie 可以用作身份验证。多个母带或“录音机”也相对容易。
然而，“从”节点需要留意主节点的关闭和启动，并采取适当的操作，例如存储监控数据，以便稍后在主节点备份时发送。
Interesting challenges, to which there are multiple solutions. The following are just my suggestions, which hopefully makes you able to better make the choice on how to write your program.
As I understand your program, you want to have one master node where you start your application. This will start the Erlang VM on the nodes in the cluster. The
poolmodule uses theslavemodule to do this, which require key-based ssh communication in both directions. It also requires that you have proper dns working.A drawback of
slaveis that if the master dies, so does the slaves. This is by design as it probably fit the original use case perfectly, however in your case it might be stupid (you may want to still collect data, even if the master is down, for example)As for the OTP applications, every node may run the same application. In your code you can determine the nodes role in the cluster using configuration or discovery.
I would suggest starting the Erlang VM using some OS facility or daemontools or similar. Every VM would start the same application, where one would be started as the master and the rest as slaves. This has the drawback of marking it harder to "automatically" run the software on machines coming up in the cluster like you could do with
slave, however it is also much more robust.In every application you could have a suitable supervision tree based on the role of the node. Removing inter-node supervision and spawning makes the system much simpler.
I would also suggest having all the nodes push to the master. This way the master does not really need to care about what's going on in the slave, it might even ignore the fact that the node is down. This also allows new nodes to be added without any change to the master. The cookie could be used as authentication. Multiple masters or "recorders" would also be relatively easy.
The "slave" nodes however will need to watch out for the master going down and coming up and take appropriate action, like storing the monitoring data so it can send it later when the master is back up.
我会研究riak_core。它提供了一个基础设施层，用于在 erlang 和 otp 本身的原始功能之上管理分布式应用程序。在riak_core下，不需要将任何节点指定为master。从 otp 意义上来说，没有一个节点是中心的，任何节点都可以接管其他失败的节点。这就是容错的本质。此外，riak_core 提供了对节点加入和离开集群的优雅处理，而无需诉诸主/从策略。
虽然这种“拓扑”去中心化很方便，但分布式应用程序通常确实需要逻辑上的特殊节点。因此，riak_core 节点可以通告它们正在提供特定的集群服务，例如，如您的用例所体现的那样，结果收集器节点。
另一个有趣的功能/架构结果是 riak_core 提供了一种通过“gossip”协议维护集群成员可见的全局状态的机制。
基本上，riak_core 包含一堆有用的代码，用于开发高性能、可靠且灵活的分布式系统。您的应用程序听起来很复杂，拥有强大的基础迟早会带来回报。
otoh，几乎还没有文档。 :(
这里有一个人谈论他用 riak_core 编写的内部 AOL 应用程序：
http:// www.progski.net/blog/2011/aol_meet_riak.html
以下是有关钢筋模板的说明：
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-March/003632.html
...这里是关于该钢筋模板的分叉的帖子：
https://github.com/rzezeski/try-try-try/blob/7980784b2864df9208e7cd0cd30a8b7c0349f977/2011/riak-core-first-multinode/README.md
...关于riak_core的讨论：
< a href="http://www.infoq.com/presentations/Riak-Core" rel="nofollow">http://www.infoq.com/presentations/Riak-Core
...riak_core 公告：
http://blog.basho.com/2010/07/30 /介绍-riak-core/
I would look into riak_core. It provides a layer of infrastructure for managing distributed applications on top of the raw capabilities of erlang and otp itself. Under riak_core, no node needs to be designated as master. No node is central in an otp sense, and any node can take over other failing nodes. This is the very essence of fault tolerance. Moreover, riak_core provides for elegant handling of nodes joining and leaving the cluster without needing to resort to the master/slave policy.
While this sort of "topological" decentralization is handy, distributed applications usually do need logically special nodes. For this reason, riak_core nodes can advertise that they are providing specific cluster services, e.g., as embodied by your use case, a results collector node.
Another interesting feature/architecture consequence is that riak_core provides a mechanism to maintain global state visible to cluster members through a "gossip" protocol.
Basically, riak_core includes a bunch of useful code to develop high performance, reliable, and flexible distributed systems. Your application sounds complex enough that having a robust foundation will pay dividends sooner than later.
otoh, there's almost no documentation yet. :(
Here's a guy who talks about an internal AOL app he wrote with riak_core:
http://www.progski.net/blog/2011/aol_meet_riak.html
Here's a note about a rebar template:
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-March/003632.html
...and here's a post about a fork of that rebar template:
https://github.com/rzezeski/try-try-try/blob/7980784b2864df9208e7cd0cd30a8b7c0349f977/2011/riak-core-first-multinode/README.md
...talk on riak_core:
http://www.infoq.com/presentations/Riak-Core
...riak_core announcement:
http://blog.basho.com/2010/07/30/introducing-riak-core/