ActiveMQ：处理消费者速度慢
关于 ActiveMQ：我有一个场景，其中有一个生产者向消费者发送小（大约 10KB）文件。虽然文件很小，但消费者需要大约 10 秒的时间来分析它们并将结果返回给生产者。我已经研究了很多，但仍然找不到以下问题的答案：
- 如何使代理将文件（完全）存储在队列中？
- 我应该使用 ObjectMessage（因为文件很小）还是 blob 消息？
- 因为消费者处理速度很慢，我应该降低他们的 prefetchLimit 还是使用循环调度策略？哪一个更好？
- 最后，在 ActiveMQ 常见问题解答中，我读到了这样的内容：“如果消费者收到一条消息并且在关闭之前没有确认该消息，则该消息将被重新传递给另一个消费者。”。所以我的问题是，ActiveMQ 是否保证只有 1 个消费者会处理该消息（因此只有 1 个给生产者的答案）？消费者何时确认消息（在默认的自动确认设置中）——接收消息并将其存储在会话中时，还是 onMessage 处理程序完成时？而且，由于消费者的处理速度非常慢，我是否应该更改一些“超时限制”，以便经纪人知道在将工作交给另一个消费者之前需要等待多长时间（这与我之前的问题有关）？
Concerning ActiveMQ: I have a scenario where I have one producer which sends small (around 10KB) files to the consumers. Although the files are small, the consumers need around 10 seconds to analyze them and return the result to the producer. I've researched a lot, but I still cannot find answers to the following questions:
- How do I make the broker store the files (completely) in a queue?
- Should I use ObjectMessage (because the files are small) or blob messages?
- Because the consumers are slow processing, should I lower their prefetchLimit or use a round-robin dispatch policy? Which one is better?
- And finally, in the ActiveMQ FAQ, I read this - "If a consumer receives a message and does not acknowledge it before closing then the message will be redelivered to another consumer.". So my question here is, does ActiveMQ guarantee that only 1 consumer will process the message (and therefore there will be only 1 answer to the producer), or not? When does the consumer acknowledge a message (in the default, automatic acknowledge settings) - when receiving the message and storing it in a session, or when the onMessage handler finishes? And also, because the consumers are so slow in processing, should I change some "timeout limit" so the broker knows how much to wait before giving the work to another consumer (this is kind of related to my previous questions)?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
不确定其他人，但这里有一些想法。
首先：我不确定您具体关心什么。 ActiveMQ 确实将消息存储在数据存储中；所有数据不需要驻留在内存中的任何单个位置（代理或客户端）。所以你实际上应该在这方面做得很好；早期版本确实要求所有 id 都需要适合内存（不确定是否已解决），但即使内存使用量也足够低，除非您有数千万条队列中消息。
至于 ObjectMessage 与 blob；原始字节数组（blob）应该是最紧凑的表示，但由于所有这些都被序列化以进行存储，因此它只影响客户端上的内存使用。预取主要有助于降低访问延迟；但考虑到它们的处理速度很慢，您可能不需要任何预取；所以是的，要么将其设置为 1 或 2，要么完全禁用。
至于保证：分布式消息队列可以保证的最好是至少一次（可能有重复）或最多一次（没有重复，可能会丢失消息）。通常最好采取至少一次，并让客户端使用客户端提供的 ID 进行重复数据删除。 JMS 规范定义了如何发送确认，因此您可以阅读有关 JMS 的更多信息；这不是 ActiveMQ 特定的。
是的，您应该将超时设置得足够高，以便工作人员通常可以完成工作，包括所有网络延迟。这可能会减慢丢失消息的重新传输速度（如果工作失败），但这对您来说可能不是问题。
Not sure about others, but here are some thoughts.
First: I am not sure what your exact concern is. ActiveMQ does store messages in a data store; all data need NOT reside in memory in any single place (either broker or client). So you should actually be good in that regard; earlier versions did require that all ids needed to fit in memory (not sure if that was resolved), but even that memory usage would be low enough unless you had tens of millions of in-queue messages.
As to ObjectMessage vs blob; raw byte array (blob) should be most compact representation, but since all of these get serialized for storage, it only affects memory usage on client. Pre-fetch mostly helps with access latency; but given that they are slow to process, you probably don't need any prefetching; so yes, either set it to 1 or 2 or disable altogether.
As to guarantees: best that distributed message queues can guarantee is either at-least-once (with possible duplicates), or at-most-once (no duplicates, can lose messages). It is usually better to take at-least-once, and make clients to de-duping using client-provided ids. How acknowledgement is sent is defiend by JMS specification so you can read more about JMS; this is not ActiveMQ specific.
And yes, you should set timeout high enough that worker typically can finish up work, including all network latencies. This can slow down re-transmit of dropped messages (if worked dies), but it is probably not a problem for you.