Zookeeper/Chubby -vs- MySql NDB
我最近一直在阅读 Paxos 论文、FLP 定理等，并评估 Apache Zookeeper 的一个项目。我还浏览了 Chubby（Google 的分布式锁定服务）以及在线提供的有关它的各种文献。我的 Zookeeper 基本用例是实现分布式系统的复制和一般协调。
我只是想知道 Zookeeper 或类似 Chubby 的分布式锁定系统带来的具体优势是什么。基本上我只是想知道为什么我不能只使用 MySQL NDB Cluster。我不断听说 MySQL 有很多复制问题。我希望一些对此主题有更多经验的人可以对此有所了解。
预先感谢..
我的要求的简单列表：
- 我有一个同质的分布式系统。
- 我需要一些方法来维持所有节点的一致状态。
- 我的系统公开了一项服务，与客户端的交互将导致我的系统的集体状态发生一些变化。
- 高可用性是一个目标，因此节点宕机一定不能影响服务。
- 我预计系统至少能够每秒处理 1000 个请求。
- 我希望系统的集体状态在大小上受到限制（基本上插入/删除将是暂时的......但在稳定状态下，我希望有大量的更新和读取）
I have been reading the Paxos paper, the FLP theorem etc. recently and evaluating Apache Zookeeper for a project. I have also been going thru Chubby (Google's distributed locking service) and the various literature on it that is available online. My fundamental usecase for Zookeeper is to implement replication and general coordination for a distributed system.
I was just wondering though, what is the specific advantage that Zookeeper or a Chubby like distributed locking system brings to the table. Basically I am just wondering why I can't just use a MySQL NDB Cluster. I keep hearing that MySQL has a lot of replication issues. I was hoping some with more experience on the subject might shed some light on it.
Thanks in advance..
A simplistic listing of my requirements :
- I have a homogeneous distributed system.
- I need some means of maintaining consistent state across all my nodes.
- My system exposes a service, and interaction with clients will lead to some change in collective state of my system.
- High availability is a goal, thus a node going down must not affect the service.
- I expect the system to service atleast a couple of 1000 req/sec.
- I expect the collective state of the system to be bounded in size (basically inserts/deletes will be transient... but in steady state, i expect lots of updates and reads)
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
这取决于您正在管理的数据类型以及您想要的规模和容错能力。
我可以从ZooKeeper的角度来回答。在开始之前我应该提到 ZooKeeper 不是 Chubby 的克隆。具体来说，它不直接进行锁定。它还在设计时考虑了不同的订购和性能要求。
在 ZooKeeper 中，系统状态的整个副本都驻留在内存中。在处理之前，大多数 ZooKeeper 服务器使用原子广播协议复制更改并同步到磁盘（使用更改日志）。因此，ZooKeeper 具有确定性的性能，只要大多数服务器正常运行，就可以容忍故障。即使出现大断电（例如电源故障），只要大多数服务器恢复在线，系统状态就会保留。 ZooKeeper 存储的信息通常被认为是系统的基本事实，因此这种一致性和持久性保证非常重要。
ZooKeeper 为您提供的其他功能与监视动态协调状态有关。临时节点允许您轻松进行故障检测和组成员身份。排序保证允许您进行领导者选举和客户端锁定。最后，手表允许您监视系统状态并快速响应系统状态的变化。
因此，如果您需要管理和响应动态配置、检测故障、选举领导者等，ZooKeeper 就是您所需要的。如果您需要存储大量数据或者需要该数据的关系模型，那么 MySQL 是更好的选择。
It depends on the kind of data you are managing and the scale and fault tolerance you are going for.
I can answer from the ZooKeeper point of view. Before starting I should mention that ZooKeeper is not a Chubby clone. Specifically it does not do locks directly. It is also designed with different ordering and performance requirements in mind.
In ZooKeeper the entire copy of system state is memory resident. Changes are replicated using an atomic broadcast protocol and synced to disk (using a change journal) by a majority of ZooKeeper servers before being processed. Because of this ZooKeeper has deterministic performance that can tolerate failures as long as a majority of servers are up. Even with a big outage, such as a power failure, as long as a majority of servers come back on line, system state will be preserved. The information stored is ZooKeeper is usually considered the ground truth of the system so such consistency and durability guarantees are very important.
The other things that ZooKeeper gives you have to do with monitoring dynamic coordination state. Ephemeral nodes allow you do to easy failure detection and group membership. The ordering guarantees allow you to do leader election and client side locking. Finally, watches allow you to monitor system state and quickly respond to changes in system state.
So if you need to manage and respond to dynamic configuration, detect failures, elect leaders, etc. ZooKeeper is what you are looking for. If you need to store lots of data or you need a relational model for that data, MySQL is a much better option.
MySQL 与 Innodb 提供了一个良好的通用解决方案，并且可能在不太昂贵的硬件上轻松满足您的性能要求。它可以在具有良好磁盘的双四核机器上轻松处理每秒数千次更新。内置的异步复制将满足您的可用性要求，但如果主数据库发生故障，您可能会丢失几秒钟的数据。当主数据库修复时，某些丢失的数据可能可以恢复，或者可以从应用程序日志中恢复：您是否可以容忍这种情况取决于您的系统的工作方式。一种损失较小但速度较慢的替代方案是使用 MySQL Innodb 在主设备和故障转移单元之间共享磁盘：在这种情况下，故障转移单元将在主设备发生故障时接管磁盘，并且不会丢失数据——只要主设备没有发生某种磁盘灾难。如果共享磁盘不可用，则可以使用 DRBD 来模拟这种情况，方法是在写入磁盘块时将其同步复制到故障转移单元：这可能会对性能产生影响。
使用 Innodb 和上面的复制解决方案之一会将您的数据复制到故障转移单元，这在很大程度上解决了恢复问题，但需要额外的胶水来重新配置您的系统以使故障转移单元联机。这通常是通过集群系统（例如 RHCS、Pacemaker 或 Heartbeat（在 Linux 上））或 Windows 上的 MS Cluster 来执行。这些系统是工具包，您需要亲自动手将它们构建成适合您环境的解决方案。但是，对于所有这些系统，当系统注意到主系统发生故障并重新配置系统以使用故障转移单元时，都会有一个短暂的中断期。这可能需要数十秒：尝试减少此时间可能会使您的故障检测系统过于敏感，并且您可能会发现系统发生不必要的故障转移。
向上，MySQL NDB 旨在减少恢复时间，并在某种程度上帮助扩展数据库以提高性能。然而，MySQL NDB的适用范围相当狭窄。该系统将关系数据库映射到分布式哈希表，因此对于涉及跨表的多个联接的复杂查询，MySQL组件和存储组件（NDB节点）之间存在相当多的流量，使得复杂查询运行缓慢。然而，适合的查询确实运行得非常快。我已经看过这个产品几次了，但是我现有的数据库太复杂而无法很好地适应，并且需要大量的重新设计才能获得良好的性能。然而，如果您正处于新系统的设计阶段，如果您能够在设计过程中牢记其限制，那么 NDB 将会很好地工作。此外，您可能会发现需要相当多的机器来提供良好的 NDB 解决方案：几个 MySQL 节点加上 3 个或更多 NDB 节点 - 尽管如果您的性能需求不是太极端，MySQL 和 NDB 节点可以共存。
即使 MySQL NDB 也无法应对整个站点丢失 - 数据中心火灾、管理错误等。在这种情况下，您通常需要另一个运行到灾难恢复站点的复制流。这通常会异步完成，以便站点间链接上的连接中断不会导致整个数据库停顿。这是由 NDB 的地理复制选项（在付费电信版本中）提供的，但我认为 MySQL 5.1 及更高版本可以本地提供此功能。
不幸的是，我对 Zookeeper 和 Chubby 知之甚少。希望其他人能够了解这些方面。
MySQL with Innodb provides a good general purpose solution, and will probably keep up with your performance requirements quite easily on not-too-expensive hardware. It can easily handle many thousands of updates per second on a dual quad-core box with decent disks. The built-in asynchronous replication will get you most of the way there for your availability requirements - but you might lose a few seconds' worth of data if the primary fails. Some of this lost data might be recoverable when the primary is repaired, or might be recoverable from your application logs: whether you can tolerate this is dependent on how your system works. A less lossy - but slower - alternative is to use MySQL Innodb with shared disk between Primary and Failover units: in this case, the Failover unit will take over the disk when the Primary fails with no loss of data -- as long as the Primary did not have some kind of disk catastrophe. If shared disk is not available, DRBD can be used to simulate this by synchronously copying disk blocks to the Failover unit as they are written: this might have an impact on performance.
Using Innodb and one of the replication solutions above will get your data copied to your Failover unit, which is a large part of the recovery problem solved, but extra glue is required to reconfigure your system to bring the Failover unit on-line. This is usually performed with a cluster system like RHCS or Pacemaker or Heartbeat (on Linux) or the MS Cluster stuff for Windows. These systems are toolkits, and you are left to get your hands dirty building them into a solution that will fit your environment. However, for all of these systems there is a brief outage period while the system notices that the Primary has failed, and reconfigures the system to use the Failover unit. This might be tens of seconds: trying to reduce this can make your failure detection system too sensitive, and you might find your system being failed over unnecessarily.
Moving up, MySQL NDB is intended to reduce the time to recovery, and to some extent help scale up your database for improved performance. However, MySQL NDB has a quite narrow range of applicability. The system maps a relational database on to a distributed hash table, and so for complex queries involving multiple joins across tables, there is quite a bit of traffic between the MySQL component and the storage components (the NDB nodes) making complex queries run slow. However, queries that fit well run very fast indeed. I have looked at this product a few times, but my existing databases have been too complicated to fit well and would require a lot of redesign to get good performance. However, if you are at the design stage of a new system, NDB would work well if you can bear its constraints in mind as you go. Also, you might find that you need quite a few machines to provide a good NDB solution: a couple of MySQL nodes plus 3 or more NDB nodes - although the MySQL and NDB nodes can co-exist if your performance needs are not too extreme.
Even MySQL NDB cannot cope with total site loss - fire at the data centre, admin error, etc. In this case, you usually need another replication stream running to a DR site. This will normally be done asynchronously so that connectivity blips on the inter-site link does not stall your whole database. This is provided with NDB's Geographic replication option (in the paid-for telco version), but I think MySQL 5.1 and above can provide this natively.
Unfortunately, I know little about Zookeeper and Chubby. Hopefully someone else can pick up these aspects.