Riak 与 GlusterFS
我需要设置一个数据存储,可以存储 PB 级别的文件(文件大多是小型 json、图像和 csv 文件,但其中一些可以是 ~100MB 的二进制文件)。
我正在研究无主且无单点故障的分布式数据存储。
我找到了 Riak 和 GlusterFS。
我想问一下大家以前用过这两个吗?
我知道那里的界面(DB/Map)非常不同。 但在我看来,他们都使用哈希和类似的分布式技术。 它们具有相似的性能、一致性和可用性吗?
I need to setup a data storage which can store PB level of files (files are mostly small json, images and csv files, but some of them can be ~100MB binary files).
I am looking into distributed data storage which is master-less and no-single-point-of-failure.
And I found Riak and GlusterFS.
I want to ask anyone of you have used both of them before?
I know that there interface (DB/Map) is very different.
But seems to me that they are both use hashing and similar distributed tech.
Will they have similar performance, consistency and availability?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
我们正在运行一个带有 Bitcask 后端的 17 个节点(24GB RAM,2T 磁盘)Riak 集群,存储大约 10 亿个 3k 对象。此设置性能良好,但非常消耗资源。我们正在考虑从 Riak 迁移到 GlusterFS,因为性能对我们来说并不是那么重要。也许使用 LevelDB 作为后端也能减轻我们的担忧。
ATM Riak的自愈特性似乎更强一些,配置也似乎更简单一些。对于您的情况,我更愿意在 GlusterFS 上存储 100MB 文件。
We are running a 17 node (24GB RAM, 2T disk) Riak cluster with a Bitcask backend, storing around 1 billion 3k objects. This setup is performant but very resource intensive. We are considering moving away from Riak to GlusterFS as performance is not that important for us. Perhaps using LevelDB as a backend would also mitigate our worries.
ATM the self healing properties of Riak seem stronger and the configuration seem a tad easier. In your case I'd be more comfortable storing 100MB files on GlusterFS.
对于普通 OSS Riak 来说,存储较大的文件(例如您提到的 100MB 文件)并不是正确的选择。
在这种情况下,你真正应该使用的是新发布的 RiakCS http://basho.com/products/riakcs / 来自芭蕉。
Storing larger files like the 100MB files you mention would not be the right choice for plain OSS Riak.
What you'd really should use in that case is the newly announced RiakCS http://basho.com/products/riakcs/ from Basho instead.
选择主要取决于要求。一般来说,如果您实际上不需要真正的文件系统(带有安装点、ACL 管理等)并且只是要以编程方式使用或提供文件,我会推荐 Riak,否则推荐 GlusterFS。
The choice depends mostly on requirements. Generally I'd recommend Riak if you do not actually need a real filesystem (with mounting points, ACLs management and so on) and just gonna use or serve files programatically, and GlusterFS otherwise.