如何实现分布式信号量?
我想要在多个进程之间共享一组有限的资源,而不是在多个服务器上运行。我想我需要的可以称为分布式信号量。
我找到了一个旧的 Perl 实现(基于 memcached)可以进行调整。我还没有完全调查过。
是否有库/组件/软件已经可以做到这一点?也许是一种算法? 故障转移怎么样?
I have a limited set of resources that I want to share between multiple processes than run on multiple servers. I guess what I need could be called a distributed semaphore.
I have found an old Perl implementation (based on memcached) that could be adapted. I have not fully investigated it yet.
Is there a library/component/software that already does that? An algorithm, perhaps?
How about failover?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
该算法称为 Paxos。还有其他算法,但它们都简化为 Paxos(或者是不正确的)。最流行的实现是 Apache Zookeeper。 Zookeeper 服务器在它们之间运行 Paxos。客户端引用命名对象并可以锁定它们等。
The algorithm for this is called Paxos. There are other algorithms, but they all reduce down to Paxos (or are incorrect). The most popular implementation of it is Apache Zookeeper. The Zookeeper servers run Paxos amongst themselves. Clients refer to named objects and can lock them etc.
我想指出,从您描述的用例来看,信号量本身可能几乎不需要分布式。所以你不一定想要一个分布式信号量,而是一个用于分布式系统的信号量。
Throttle 可能符合此要求:https://github.com/pacman82/throttle
I want to point out that from the use case you describe there is probably little need for the semaphore itself to be distributed. So you do not necessarily want a distributed semaphore, but a semaphore a for distributed system.
Throttle might fit this bill: https://github.com/pacman82/throttle
为了实现分布式信号量,您需要在节点集群中定义一个权限,然后一旦选择它,您需要将请求集中到它。处理锁请求很容易,选择领导者则更复杂。
Paxos 将为您解决这个问题。我刚刚更新了 wikipedia 页面,因为算法描述不完整且具有误导性。
In order to implement a distributed semaphore, you need to define an authority amongst a cluster of node, then once it is selected, you need to centralize requests to it. Handling lock requests is a piece of cake, selecting the leader is more complicated.
Paxos will solve that for you. I have just updated the wikipedia page, because the algorithm description was incomplete and misleading.
有一个更简单的类似于paxos的算法,叫做raft:
http://raftconsensus.github.io/
there is a simpler algorithm similar to paxos, it's called raft:
http://raftconsensus.github.io/
我想建议基于领事会话实现分布式信号量。 Consul 中的会话代表具有非常特定语义的合约。
Consul 提供的契约是,在以下任何情况下,会话将失效:
时会话失效,被销毁并且不能再使用。
详细指南:https://www.consul.io/docs/internals/sessions.html
I'd like to suggest implement distributed semaphore based on consul sessions. A session in Consul represents a contract that has very specific semantics.
The contract that Consul provides is that under any of the following situations, the session will be invalidated:
When a session is invalidated, it is destroyed and can no longer be used.
Detailed guide: https://www.consul.io/docs/internals/sessions.html