如何使用ZooKeeper设计任务分配
我计划编写一个具有分布式工作进程的应用程序。其中之一将是领导者，它将任务分配给其他进程。设计 Leader 选举过程非常简单：每个过程都尝试在同一路径中创建一个临时节点。谁成功了，谁就成为领导者。
现在，我的问题是如何设计均匀分配任务的流程？有什么食谱吗？
我将详细说明一下环境设置：
假设有 10 台工作机，每台运行一个进程，其中一台成为领导者。任务在队列中提交，Leader 接收它们并分配给工作人员。每当提交任务时，工作进程都会收到通知。
I am planning to write an application which will have distributed Worker processes. One of them will be Leader which will assign tasks to other processes. Designing the Leader elelection process is quite simple: each process tries to create a ephemeral node in the same path. Whoever is successful, becomes the leader.
Now, my question is how to design the process of distributing the tasks evenly? Any recipe for this?
I'll elaborate a little on the environment setup:
Suppose there are 10 worker maschines, each one runs a process, one of them become leader. Tasks are submitted in the queue, the Leader takes them and assigns to a worker. The worker processes gets notified whenever a tasks is submitted.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
我不确定我是否理解您的领导者选举算法，但推荐的实现方法是使用顺序临时节点并使用 http://zookeeper.apache.org/doc/r3.3.3/recipes.html#sc_leaderElection 解释了如何避免“羊群”效应。
任务的分发可以通过简单的分布式队列来完成，并不严格需要 Leader。生产者将任务排队，消费者监视任务节点 - 触发的监视将导致消费者接受任务并删除关联的 znode。对失败的消费者重新排队任务时需要考虑某些边缘条件。 http://zookeeper.apache.org/doc/r3.3.3/recipes .html#sc_recipes_Queues
I am not sure I understand your algorithm for Leader election, but the recommended way of implementing this is to use sequential ephemeral nodes and use the algorithm at http://zookeeper.apache.org/doc/r3.3.3/recipes.html#sc_leaderElection which explains how to avoid the "herd" effect.
Distribution of tasks can be done with a simple distributed queue and does not strictly need a Leader. The producer enqueues tasks and consumers keep a watch on the tasks node - a triggered watch will lead the consumer to take a task and delete the associated znode. There are certain edge conditions to consider with requeuing tasks from failed consumers. http://zookeeper.apache.org/doc/r3.3.3/recipes.html#sc_recipes_Queues
我会推荐这本书的
示例：Master-Worker应用程序部分ZooKeeper分布式进程协调http://shop.oreilly.com/product/0636920028901.do该示例演示了如何使用 znode 和常见的 Zookeeper 命令将任务分发给工作人员。
I would recommend the section
Example: Master-Worker Applicationof this bookZooKeeper Distributed Process Coordinationhttp://shop.oreilly.com/product/0636920028901.doThe example demonstrates to distribute tasks to worker using znodes and common zookeeper commands.
考虑使用参与者单例服务模式。例如，在 Scala 中，有 Akka 可以用更少的代码解决此类问题。
Consider using an actor singleton service pattern. For example, in Scala there is Akka which solves this class of problem with less code.