ZeroMQ、RabbitMQ 和 Apache Qpid 的性能比较
我的应用程序需要高性能消息总线，因此我正在评估 ZeroMQ、RabbitMQ 和 Apache Qpid 的性能。为了测量性能，我正在运行一个测试程序，该程序使用其中一个消息队列实现发布 10,000 条消息，并在同一台计算机中运行另一个进程来使用这 10,000 条消息。然后我记录发布的第一条消息和收到的最后一条消息之间的时间差。
以下是我用于比较的设置。
RabbitMQ：我使用了“扇出”类型的交换器和具有默认配置的队列。我使用 RabbitMQ C 客户端库。ZeroMQ：我的发布者使用ZMQ_PUSH套接字发布到tcp://localhost:port1，我的代理侦听tcp://localhost :port1并将消息重新发送到 tcp://localhost:port2，我的消费者使用ZMQ_PULL套接字监听tcp://localhost:port2。我在 ZeroMQ 中使用代理而不是点对点通信，以使性能比较与使用代理的其他消息队列实现公平。QpidC++ 消息代理：我使用了“扇出”类型的交换器和具有默认配置的队列。我使用了 Qpid C++ 客户端库。
以下是性能结果：
RabbitMQ：接收 10,000 条消息大约需要 1 秒。ZeroMQ：接收10000条消息大约需要15毫秒。Qpid：接收10000条消息大约需要4秒。
问题：
- 有人在消息队列之间进行过类似的性能比较吗？然后我喜欢将我的结果与你的进行比较。
- 有什么方法可以调整
RabbitMQ或Qpid以使其性能更好吗？
注意：
测试是在分配了两个处理器的虚拟机上完成的。对于不同的硬件，结果可能会有所不同，但我主要对 MQ 产品的相对性能感兴趣。
I need a high performance message bus for my application so I am evaluating performance of ZeroMQ, RabbitMQ and Apache Qpid. To measure the performance, I am running a test program that publishes say 10,000 messages using one of the message queue implementations and running another process in the same machine to consume these 10,000 messages. Then I record time difference between the first message published and the last message received.
Following are the settings I used for the comparison.
RabbitMQ: I used a "fanout" type exchange and a queue with default configuration. I used the RabbitMQ C client library.ZeroMQ: My publisher publises totcp://localhost:port1withZMQ_PUSHsocket, My broker listens ontcp://localhost:port1and resends the message to tcp://localhost:port2 and my consumer listens ontcp://localhost:port2usingZMQ_PULLsocket. I am using a broker instead of peer to to peer communication inZeroMQto to make the performance comparison fair to other message queue implementation that uses brokers.QpidC++ message broker: I used a "fanout" type exchange and a queue with default configuration. I used the Qpid C++ client library.
Following is the performance result:
RabbitMQ: it takes about 1 second to receive 10,000 messages.ZeroMQ: It takes about 15 milli seconds to receive 10,000 messages.Qpid: It takes about 4 seconds to receive 10,000 messages.
Questions:
- Have anyone run similar performance comparison between the message queues? Then I like to compare my results with yours.
- Is there any way I could tune
RabbitMQorQpidto make it performance better?
Note:
The tests were done on a virtual machine with two allocated processor. The result may vary for different hardware, however I am mainly interested in relative performance of the MQ products.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
不知道你为什么要这样做——如果你唯一关心的是性能，那么就没有必要让竞争环境变得公平。如果你不关心持久性、过滤等，那为什么要付出代价呢？
我对在虚拟机上运行基准测试也非常谨慎——有很多额外的层可能会以不明显的方式影响结果。 （当然，除非您计划在虚拟机上运行真实的系统，在这种情况下这是一个非常有效的方法）。
Not sure why you want to do that -- if the only thing you care about is performance, there is no need to make the playing field level. If you don't care about persistence, filtering, etc. then why pay the price?
I'm also very leery of running benchmarks on VM's -- there are a lot of extra layers that can affect the results in ways that are not obvious. (Unless you're planning to run the real system on VM's, of course, in which case that is a very valid method).
我测试过 c++/qpid，
我在两台不同的机器之间每秒发送 50000 条消息，很长一段时间没有排队。
我没有使用扇出，只是一个简单的交换（非持久消息）
您使用持久消息吗？
您正在解析消息吗？
我想不会，因为 0MQ 没有消息结构。
如果代理主要处于空闲状态，则您可能尚未在发送方和接收方上配置预取。这对于发送许多消息非常重要。
I've tested c++/qpid
I sent 50000 messages per second between two diferent machines for a long time with no queuing.
I didn't use a fanout, just a simple exchange (non persistent messages)
Are you using persistent messages?
Are you parsing the messages?
I suppose not, since 0MQ doesn't have message structs.
If the broker is mainly idle, you probably haven't configured the prefetch on sender and receptor. This is very important to send many messages.
我们将 RabbitMQ 与 SocketPro (http://www.udaparts.com/) 持久消息队列进行了比较，网址为该网站 http://www.udaparts.com/document/articles/fastsocketpro.htm 包含所有源代码。以下是我们从 RabbitMQ 获得的结果：
同一台机器入队和出队：
网络带宽 100 mbps 的跨机器入队和出队：
We have compared RabbitMQ with our SocketPro (http://www.udaparts.com/) persistent message queue at the site http://www.udaparts.com/document/articles/fastsocketpro.htm with all source codes. Here are results we obtained for RabbitMQ:
Same machine enqueue and dequeue:
Cross-machine enqueue and dequeue with network bandwidth 100 mbps:
我们开发了一个基于 ZeroMQ 的开源消息总线 - 我们最初这样做是为了取代 Qpid。它是无代理的，因此这不是一个完全公平的比较，但它提供了与代理解决方案相同的功能。
我们的主要性能数据是两台机器之间每秒 140K 条消息，但您可以在此处查看更多详细信息： https://github.com/Abc-Arbitrage/Zebus/wiki/Performance
We've developed an open source message bus built on top of ZeroMQ - we initially did this to replace Qpid. It's brokerless so it's not a totally fair comparison but it provides the same functionality as brokered solutions.
Our headline performance figure is 140K msgs per second between two machines but you can see more detail here: https://github.com/Abc-Arbitrage/Zebus/wiki/Performance
尝试在发送方和接收方上配置预取值，例如 100。仅预取发送方是不够的
Try to configure prefetch on sender and receptor with a value like 100. Prefetching just sender is not enough
RabbitMQ 可能正在对这些消息进行持久化。我认为您需要设置消息优先级或消息中的其他选项以不进行持久化。届时性能将提高 10 倍。您预计每秒至少有 100K 条消息通过 AMQP 代理。在 OpenAMQ 中，我们的性能高达 300K 消息/秒。
AMQP 是为了速度而设计的（例如，它不会解包消息来路由它们），但 ZeroMQ 在关键方面设计得更好。例如，它通过在没有代理的情况下连接节点来删除一跳；它比任何 AMQP 客户端堆栈都具有更好的异步 I/O 性能；它执行更积极的消息批处理。也许构建 ZeroMQ 的 60% 时间都花在了性能调优上。这是非常辛苦的工作。它的速度并不是偶然的。
我想做但太忙的一件事是在 ZeroMQ 之上重新创建一个类似 AMQP 的代理。这里有第一层：http://rfc.zeromq.org/spec:15。整个堆栈的工作方式有点像 RestMS，传输和语义分为两层。它将提供与 AMQP/0.9.1 大致相同的功能（并且在语义上可互操作），但速度明显更快。
RabbitMQ is probably doing persistence on those messages. I think you need to set the message priority or another option in messages to not do persistence. Performance will improve 10x then. You should expect at least 100K messages/second through an AMQP broker. In OpenAMQ we got performance up to 300K messages/second.
AMQP was designed for speed (e.g. it does not unpack messages in order to route them) but ZeroMQ is simply better designed in key ways. E.g. it removes a hop by connecting nodes without a broker; it does better asynchronous I/O than any of the AMQP client stacks; it does more aggressive message batching. Perhaps 60% of the time spent building ZeroMQ went into performance tuning. It was very hard work. It's not faster by accident.
One thing I'd like to do, but am too busy, is to recreate an AMQP-like broker on top of ZeroMQ. There is a first layer here: http://rfc.zeromq.org/spec:15. The whole stack would work somewhat like RestMS, with transport and semantics separated into two layers. It would provide much the same functionality as AMQP/0.9.1 (and be semantically interoperable) but significantly faster.
嗯，当然 ZeroMQ 会更快，它被设计为并且没有其他两个提供的很多基于代理的功能。 ZeroMQ 网站 对代理与无代理消息传递以及缺点和缺点进行了精彩的比较。两者的优点。
RabbitMQ 博客：
（我也喜欢上面的 RabbitMQ 帖子，因为它还讨论了将 ZeroMQ 与 RabbitMQ 结合使用）
因此，我想说的是，您应该决定最适合您要求的技术。如果唯一的要求是速度，ZeroMQ。但是，如果您需要其他方面，例如消息持久性、过滤、监控、故障转移等，那么您就需要开始考虑 RabbitMQ 和 RabbitMQ。 Qpid。
Hmm, of course ZeroMQ will be faster, it is designed to be and does not have a lot of the broker based functionality that the other two provide. The ZeroMQ site has a wonderful comparison of broker vs brokerless messaging and drawbacks & advantages of both.
RabbitMQ Blog:
(I also like the above RabbitMQ post above as it also talks about using ZeroMQ with RabbitMQ)
So, what I'm trying to say is that you should decide on the tech that best fits your requirements. If the only requirement is speed, ZeroMQ. But if you need other aspects such as persistence of messages, filtering, monitoring, failover, etc well, then that's when u need to start considering RabbitMQ & Qpid.