Apache Camel - 将消息路由到 JPA 端点 - 保持订单但根据类别标头分成池
我们有一个简单的项目，它从多个端点（代理）获取大量消息。这些代理都输出相同格式的消息（要放置在数据库中的实体对象）。所有代理都写入同一个队列，我们使用它们并通过 JPA 将它们发送到数据库。
因此，本质上，系统有一组生产者将消息写入一个队列。队列是单线程的，只接收消息并将其转储到数据库中。
这里的问题是这种方法很慢。 Camel 中是否有任何功能（例如重新排序）可以用来根据消息的来源拆分这些消息。因此，虽然来自 Agent1 的消息需要按照创建的顺序进行持久化，但来自 Agent2 的消息是独立的，因此它们不应该按照 Agent1 的消息顺序等待。对于两个代理来说，这是一个简单的问题，因为我们只需创建两个队列，每个代理一个。我们有许多代理，因此我们需要一个可扩展的解决方案。
是否有任何模式可以在骆驼中本地完成此操作？我们可以编写自己的保留队列，该队列与代理名称同步，并且只将一条消息放入多线程 JPA 写入队列，但这将是一种迂回的方式，因为我们需要设置从队列到 jpa 骆驼路由的回调，或者我们不会使用骆驼，而只是通过我们自己的管理器来完成（并不是说这会很复杂，但如果我们可以使用骆驼或其他东西来完成这一切，那就太好了并且不必重新发明轮子说话）。
We have simple project which takes a number of messages from a number of endpoints(agents). These agents all output the same format message (an entity object to be placed in a database). All the agents write to the one queue and we consume these and send them to a database via JPA.
So essentially the system has a collection of producers writing messages to one queue. The queue is single threaded and just takes the messages as they come and dumps them into the database.
The issue here is this method is slow. Is there any functionality in Camel (like re-sequencing) that we could use to split out these messages based on their source. So while the messages from Agent1 need to be persisted in the order they are created, the messages from Agent2 are separate so they should not wait on the order of Agent1's messages. For two agents this is an easy problem as we just create two queues, one for each agent. We have a number of agents, so we need a solution that can scale.
Are there any patterns to accomplish this natively in camel? We could write our own holdout queue which syncs on the Agent name and only ever puts one message through to a multi-threaded JPA write queue, but this would be a bit of a round-about way to do things as we would need to either setup a callback from the queue to jpa camel route or we would not use camel and just do it via our own manager (not that this would be complex, but it would be great if we could do this all using Camel or something else out there and not have to reinvent the wheel so to speak).
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
如果您的源是 JMS 消息队列，那么请在此处查看
有关此内容的消息组 Apache ActiveMQ 文档：
http://activemq.apache.org/message-groups.html
以及此常见问题解答
http://activemq.apache.org/how-do -i-preserve-order-of-messages.html
基本上你可以使用JMSXGroupID JMS属性来标记代理id，例如agent1，agent2等。
然后你可以在JMS消息队列上有并发消费者，这可以运行基于 JMSXGroupID 并行。但仍保留每个组内的顺序。这意味着您可以并行写入 JPA agent1、agent2、...agentN。
If you source is a JMS message queue, then take a look at message groups
Apache ActiveMQ documents about this here:
http://activemq.apache.org/message-groups.html
And this FAQ
http://activemq.apache.org/how-do-i-preserve-order-of-messages.html
Basically you can use the JMSXGroupID JMS property to mark the agent id, eg agent1, agent2, etc.
Then you can have concurrent consumers on the JMS message queue, which can run in parallel based on the JMSXGroupID. But still preserve ordering within each group. That means you can in parallel write to JPA agent1, agent2, ... agentN.