Java/JMS - 处理失败场景的消息数量
我有一个要求，我必须处理大量消息。这些消息将由另一个进程插入到数据库表中。我所要做的就是检查数据库中的新消息并将其发送给相应的客户，无论是电子邮件还是 http，具体取决于他们的配置。在任何给定时间，消息数量可能有数千条，而客户数量约为 1000 个。
我计划按照生产者消费者的工作方式进行设计。就像生产者线程轮询数据库中的新消息并将它们放入队列中一样，工作线程读取这些消息并进行处理。
乍一看，JMS 似乎是满足此需求的正确解决方案。但我正在寻找是否有更好的选择，例如 ExecutorService，在以下场景中使用适合此要求的线程池。
- 如果一条消息发送失败，我必须至少在 24 小时内重试多次。
- 如果某个客户的一条消息传递失败，那么其他消息传递也很可能会失败。因此，我应该尝试在为该客户处理下一条消息之前发送第一条消息。
这意味着，如果我有一个队列来存放所有客户的所有消息，那么如果一条消息失败，其他消息将不会得到处理。
有人可以给我建议如何最好地处理这个问题吗？
提前致谢。
I have a requirement where I have to process number of messages. These messages will be inserted into a DB table by another process. All I have to do is to check for new messages in DB and send it to corresponding customer either email or http, depending on their configuration. The number of messages could be several thousands at any given time and the number of customers are around 1000.
I am planning to design this the way producer consumer works. Like producer thread polls DB for new messages and puts them into Queue and worker threads read these messages and processes.
At first it looked like JMS is the right solution for this requirement. But am seeking if any better alternatively like ExecutorService, using Threadpool suitable for this requirement given the following scenarios.
- if one message delivery fails, I will have to retry it several times at least for 24 hours.
- if one message delivery fails for a customer, all chances that other message delivery will also fail. So, I should try to send the first message before I process next message for that customer.
That means if I have one queue for all messages for all customers, then if one message fails, others will not be processed.
Can someone give me suggestion on how best I can handle this?
Thanks in advance.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
您应该认真看看 Apache Camel。它处理所有的线程、生产和操作。为您使用大量内置组件（SQL、JMS、SMTP、HTTP、文件等），并且您关于消费者/生产者的术语与 Camel 集成视图相匹配。
实现像您描述的那样简单：
看看 端点列表
You should take a serious look at Apache Camel. It handles all of the threading, producing & consuming for you with a large number of built in components (SQL, JMS, SMTP, HTTP, File, etc) and your terminology around Consumers/Producers matches the Camel view of integration.
Implementing something like you describe is as easy as:
Have a look at the list of endpoints
使用通用 JMS 实现重试逻辑相对容易：无事务、为消息分配重试属性、每次重试时递增该属性、重试时在侦听器中暂停、限制耗尽后退出。
每个客户的队列是最简单的配置。动态队列可以帮助您处理客户：动态为每个客户创建一个队列和一个侦听器。
实际上，可靠地轮询数据库可能会给您带来更多挑战。
Retry logic is relatively easy to implement with generic JMS: no transactions, assign message a retry attribute, increment it with each retry, pause in listener when retrying, quit after limit is exhausted.
Queue per customer is the simplest configuration. Dynamic queues may help you with customers: create a queue and a listener for each customer dynamically.
Actually polling database reliably may present more challenge for you.