用于存储 Erlang 统计数据的最佳数据库
我需要选择一个数据库来存储统计数据（实际上这是一系列时间戳值数据）。我知道几乎任何数据库都可以处理这个问题，但有几个要求：
- 它应该很快；
- 它应该能够处理很多 数据（数十GB）并拼接它 快速地;
- 它应该有一个稳定的、可维护的 以及方便的 Erlang 接口；
- 它应该可以从 Python 获得；
- 它应该能够做出一些东西 就像那个叫“上限”的东西 mongodb 中的集合：集合 具有上限大小，具有旧数据 如果大小达到则被重写 限制。
我想到了 mongo，但 emongo 似乎有点死了——最后一次提交是 7 个月前进行的。
I need to choose a database for storing statistical data (in fact this is a series of timestamp-value data). I understand that virtually any database can handle this, but there are a couple of requirements:
- it should be fast;
- it should be able to handle A LOT of
data (10s of gigabytes) and splice it
fast; - it should have a stable, maintained
and handy interface to Erlang; - it should be available from Python;
- it should be able to make something
like the thing named "capped
collections" in mongodb: collection
with the capped size, with old data
being rewritten if the size reach the
limit.
I thought about mongo, but emongo seems to be a little dead - the last commit was made 7 months ago.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
Riak 可能是一个不错的选择（这里有一个 Riak 与 MongoDB 的比较）。它是用 Erlang 编写的，具有分布式、容错性和线性扩展性。它有用于 Erlang、Javascript、Java、PHP、Python、Ruby 的客户端。一个 REST 接口、一个 protobuf 接口和许多其他好东西（MapReduce、链接、复制、提交前/提交后挂钩，...）。它是开源的，由 Basho 创建并维护。 Basho 拥有 Riak 的商业产品以及一些额外的功能（如多站点复制、SNMP 监控）等）但操作系统版本有很大的价值。
根据您的需求，结合几种技术可能是有意义的。例如，您可以在系统前面使用 Redis 之类的内存存储来提高速度，并使用 Riak 来保存数据。 Redis + Riak 是一个非常甜蜜的堆栈。
Riak may be a good choice (here's a Riak comparison to MongoDB). It's written in Erlang, is distributed, fault tolerant and scales linearly. It has clients for Erlang, Javascript, Java, PHP, Python, Ruby. A REST interface, a protobuf interface and so many other goodies (Map Reduce, links, replication, pre/post commit hooks, ...). It's open source and is created maintained by Basho. Basho has commercial offering of Riak as well with some extra features (like multi-site replication, SNMP monitoring, etc) but there's awsome value in the OS version.
Depending on your needs it may make sense to combine a couple of technologies. For example you could front your system with an in memory store like Redis for speed and use Riak to persist the data. Redis + Riak is a pretty sweet stack.
我认为 postgresql 和 pgsql 驱动程序将是您的最佳解决方案。
I think postgresql and pgsql driver it will be best solution for you.
磁盘上的文件经过旋转后可以很好地满足您的需求。关键是您不想快速搜索数据。
Files on disk, rotated, will serve your demands fine. The point is you don't want to search data quickly.
redis 是一个相当有力的竞争者。
当前唯一的限制是数据集的大小，它必须完全存储在内存中或使用 VM 方法，其中只有键空间必须适合内存（但是，为实际数据留出一点空闲空间）很好）但启动时间非常慢。
开发人员 Antirez 正在将后端重写为名为 diskstore 的内容 这应该可以解决您的问题。虽然还没有出炉，但我对这个项目很有信心。
关于上限集合，Redis 没有直接的方法来处理它。但是 LTRIM 函数可以帮助你。
redis is quite a close contender.
The only current limitation is the size of the dataset, which has to be either store in full in memory or use the VM method, in which only the key space has to fit in memory (however a bit of spare room for actual data would be nice) but has a very slow startup time.
Antirez, the developer, is rewriting the backend into something called diskstore which should solve your issue. It's not baked yet, but I have a lot of confidence in this project.
About the capped collections, redis does not have a direct way for handling that. But the LTRIM function can help you out.