RabbitMQ / ActiveMQ 或 Redis 超过 250,000 条消息/秒
尽管 Redis 和消息队列软件通常用于不同的目的,但我想问一下在以下用例中使用 Redis 的优点和缺点:
- 一组事件收集器将传入消息写入键/值。消费者获取和删除已处理密钥的
- 负载从 100k msg/s 开始,并在短时间内(例如几个月)超过 250k,目标是实现百万条 msg/s
- 持久性不是严格要求的。在故障期间丢失非日志消息
- 是可以的,性能非常重要(因此,处理负载所需的系统数量)
- 消息不必按照它们到达的顺序进行处理,
您知道选择 Redis 而不是传统的用例消息队列软件 ?或者你会考虑别的吗?
注意:我也看到了这个,但没有帮助: 实时应用程序新手 - Node.js JS + Redis 或 RabbitMQ ->客户端/服务器如何?
谢谢
Eventhough redis and message queueing software are usually used for different purposes, I would like to ask pros and cons of using redis for the following use case:
- group of event collectors write incoming messages as key/value . consumers fetch and delete processed keys
- load starting from 100k msg/s and going beyond 250k in short period of time (like months) target is to achieve million msg/s
- persistency is not strictly required. it is ok to lose non-journaled messages during failure
- performance is very important (so, the number of systems required to handle load)
- messages does not have to be processed in the order they arrive
do you know such use cases where redis chosen over traditional message queueing software ? or would you consider something else ?
note: I have also seen this but did not help:
Real-time application newbie - Node.JS + Redis or RabbitMQ -> client/server how?
thanks
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
鉴于您的要求,我会尝试 Redis。它将比其他解决方案表现更好,并为您提供对持久性特征更细粒度的控制。根据您使用的语言,您也许能够使用分片 Redis 集群(您需要支持一致哈希的 Redis 绑定 - 并非全部都支持)。这将使您能够扩展到您指定的卷。在一些基本测试中,我在笔记本电脑上看到了 10k/秒。
如果您需要队列语义,您可能需要使用 Redis 中的列表操作(LPUSH 用于写入,BRPOP 用于读取)。
我有一个前客户,去年春天在生产中将 Redis 部署为消息队列,他们对此非常满意。
Given your requirements I would try Redis. It will perform better than other solutions and give you much finer grained control over the persistence characteristics. Depending on the language you're using you may be able to use a sharded Redis cluster (you need Redis bindings that support consistent hashing -- not all do). This will let you scale out to the volume you indicated. I've seen 10k/sec on my laptop in some basic tests.
You'll probably want to use the list operations in Redis (LPUSH for writes, BRPOP for reads) if you want queue semantics.
I have a former client that deployed Redis in production as a message queue last spring and they've been very happy with it.