适用于少量节点的冗余分布式文件系统

发布于 2024-09-15 02:12:59 字数 387 浏览 1 评论 0原文

我需要以完全冗余的方式在不同位置的两个节点之间共享文件系统,以便在两个节点中运行的程序应该看到整个文件系统。

这种情况是数据文件可以存储在任何一个节点中,然后在另一个节点中运行的程序可能已被第三方通知该文件的存在,并且它期望该文件存在。即使使用自制技巧(例如 inotify)加速,rsync 镜像也有太多延迟。我们肯定需要一个共享文件系统。

需要冗余是为了确保在另一个数据中心无法访问时我们可以继续使用单个节点。

只有两三个节点来共享文件,可能lustre或HDFS有点过分了,但可以考虑,Ceph似乎太年轻,XtreemFS有困难,DRBD听起来不错(而且很小),但我担心它可能会遇到问题跳过站点之间的高网络延迟。

我错过了一些可能性吗?是否有人确实进行了此设置,跨两个数据中心共享文件系统?

I need to keep a filesystem shared between two nodes in different locations, in a way fully redundant, so that programs running in both nodes should see the whole filesystem.

The situation is that a datafile could be stored in any of the nodes, then a program running in the other could have been notified by a third party about the existence of the file, and it expects it to exist. A rsync mirroring has too much latency, even if accelerated with house-made tricks (inotify, for one). Definitely we need a shared filesystem.

Redundandy is wanted in order to be sure that we can keep going with a single node if the other datacenter becomes unreacheable.

Having only two or three nodes to share the files, probably lustre or HDFS is overkill, but it could be considered, Ceph seems too young, XtreemFS has difficulties, and DRBD sounds nice (and small) but I am afraid it could have problems to jump over high network latencies, between sites.

Am I missing some posibility? Is someone actually having this setup, sharing a filesystem across two data centers?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

满栀 2024-09-22 02:12:59

我推荐优秀的 LizardFSGfarmFS

Ceph 太复杂,因此非常不可靠且脆弱;它主要用于块存储,Ceph 文件系统非常慢。我建议避免使用 Ceph,因为它的复杂性和对数据完整性的严重忽视。
XtreemFS 很慢(可能是因为 Java);它的构建系统很混乱;它存在数据完整性问题,并且依赖于非免费/未维护的 .JAR。

MARS 可能是 DRBD 的更好替代方案,但我记得 MARS 需要修改内核及其 DKMS 版本尚未发布。

I recommend excellent LizardFS and GfarmFS.

Ceph is too complex hence very unreliable and fragile; it is mostly used for block storage and the Ceph file system is very slow. I recommend to avoid Ceph due to its complexity and gross disregard to data integrity.
XtreemFS is slow (probably because of Java); its build system is messy; it have problems with data integrity and depends on non-free/unmaintained .JARs.

MARS could be a better alternative to DRBD but as I recall MARS needs modified kernel and its DKMS version is yet to be released.

心的憧憬 2024-09-22 02:12:59

也许Twin Peaks Software开发的镜像文件系统(MFS)可以满足
你需要吗?
http://TwinPeakSoft.com/

Maybe Mirror File System (MFS) developed by Twin Peaks Software can meet
your need?
http://TwinPeakSoft.com/

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文