是否可以在 Erlang 中实现 Pregel 而无需超级步骤?

发布于 2024-12-05 14:32:57 字数 268 浏览 1 评论 0原文

假设我们使用 Erlang 实现 Pregel 。为什么我们实际上需要超级步骤?仅将消息从一个主管发送到代表节点的进程不是更好吗?他们可以将计算函数应用到自己身上,互相发送消息,然后向主管发送“完成”消息。

Pregel 并发 Erlang 实现中超级步骤的全部目的是什么?

Let's say we implement Pregel with Erlang. Why do we actually need supersteps? Isn't it better to just send messages from one supervisor to processes that represent nodes? They could just apply the calculation function to themselves, send messages to each other and then send a 'done' message to the supervisor.

What is the whole purpose of supersteps in concurrent Erlang implementation of Pregel?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

晚风撩人 2024-12-12 14:32:57
  • Pregel 模型所拥护的 SuperStep 概念可以被视为并行执行实体的某种障碍。在每个超级步骤结束时,每个工作人员将其状态刷新到持久存储中。
  • 该算法在每个 SuperStep 的末尾设置检查点,以便在发生故障时,当新节点必须接管故障对等点的功能时,它有一个起点。 Pregel 保证,由于节点的数据在 SuperStep 启动之前就已刷新到磁盘,因此它可以可靠地从该点开始。
  • 它在某种程度上也标志着算法的“进步”。可以为预凝胶算法/作业提供“最大超级步骤数”,之后算法应终止。

您在问题中指定的内容(关于主管向工作人员发送计算函数并等待“完成”)绝对可以实现(尽管我不认为与 OTP 打包的当前主管可以开箱即用地执行类似的操作),但我想SuperStep 的概念只是 Pregel 模型的一个要求。另一方面,如果您正在实现类似并行映射器的东西(就像乔在他的书中实现的那样),您将不需要超级步骤/

  • The SuperStep concept as espoused by the Pregel model could be viewed as sort of a Barrier for parallel-y executing entities. At the end of each superstep, each worker, flushes it state to the persistent store.
  • The algorithm is check-pointed at the end of each SuperStep so that in case of failure, when a new node has to take over the function of a failed peer, it has a point to start from. Pregel guarantees that since the data of the node has been flushed to disk before the SuperStep started, it can reliably start from exactly that point.
  • It also in a way signifies "progress" of the algorithm. A pregel algorithm/job can be provided with a "max number of supersteps" after which the algorithm should terminate.

What you specified in your question (about superisors sending worker a calculation function and waiting for a "done") can definitely be implemented (although I dont think the current supervisor packaged with OTP can do stuff like that out of the box) but I guess the concept of a SuperStep is just a requirement of a Pregel model. If on the other hand, you were implementing something like a parallel mapper (like what Joe implements in his book) you wont need supersteps/

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文