如何“复制”一条 JMS 消息发送到 2 个目的地?

发布于 2024-08-08 05:58:58 字数 2138 浏览 1 评论 0原文

我有一个要求,即客户端发送的单个 JMS 消息必须可靠地(恰好一次)传递到两个系统。这两个系统没有启用 HA,因此我提出的最佳建议是:

  1. 创建客户端发布的单个队列

  2. 设置两个“中间” ” 队列

  3. 使用自定义“DuplicatorMDB”将从客户端队列读取消息并将它们发布到同一事务中的两个队列。

client->JMSDQ->DuplicatorMDB->Q1->MDB->System1
                           \->Q2->MDB->System2

现在有类似的功能吗?如果一个或两个后端系统出现故障,平衡系统以保持稳定的正确方法是什么?

应用程序服务器是WebLogic 10。

我不能为此使用主题,因为在集群中主题会导致太多消息重复。如果我们有 2 个实例,那么主题将如下所示:

 
client->Topic-->MDB1@server1->System1
           | \->MDB2@server1->System2 
           \---->MDB1@server2->System1 
            \--->MDB2@server2->System2 

因此,每条消息将被传送到 System1 两次,到 System2 两次,如果集群中有 8 个服务器,则每条消息将被传送 8 次。这是我真正想避免的......

最后我有一些时间来测试它,这是我观察到的: 集群中有 2 个节点。 2 个 JMS 服务器:jms1 在节点 1 上,jms2 在节点 2 上。 分布式主题 dt.具有持久订阅和 jms-client-id=durableSubscriber 的 MDB。启动系统:0 条消息,mdb@node1 已启动,mdb@node2 尝试定期连接,但无法连接,因为“客户端 id,durableSubscriber,正在使用”。正如预期的那样。

已发送 100 条消息: jms1@dt 消息当前 = 0,消息总数 = 100,消费者当前 = 1 我可以看到node1处理了100条消息。
jms2@dt 消息当前 = 100,消息总数 = 100,消费者当前 = 1 即“重复”消息在主题中待处理。

另外发送了 100 条消息,其中 100 条已在节点 1 上处理,200 条在节点 2 上待处理。

重新启动node1,mdb@node2重新连接到dt并开始处理“待处理”消息。在节点 2 上处理了 200 条消息。

Node1启动后,mdb@node1无法连接到dt,而mdb@node2可以连接。

jms1@dt 消息当前 = 0,消息总数 = 0,消费者当前 = 0
jms2@dt messages current=0, messages total = 200,consumers current = 1

再发送100条消息,我看到所有100条消息都在node2上处理并在node1上丢弃。

jms1@dt 消息当前 = 0,消息总数 = 100,消费者当前 = 0
jms2@dt 消息当前=0,消息总数=300,消费者当前=1

现在我重新启动node2,mdb@node1 重新连接到dt。重新启动后,mdb@node2 重新连接到 dt,mdb@node1 与 dt 断开连接。

jms1@dt 消息当前 = 0,消息总数 = 100,消费者当前 = 1
jms2@dt 消息当前 = 0,消息总数 = 0,消费者当前 = 1

我发送了 100 条消息,所有消息都在节点 2 上处理并存储在节点 1 上的主题中:

jms1@dt 消息当前 = 100,消息总数 = 200,消费者当前 = 1
jms2@dt messages current=0, messages total = 0,consumers current = 1

然后我关闭了 node2,在 mdb@node1 重新连接到该主题后,我看到节点 1 上正在处理 100 条“待处理消息”。

所以结果是: 我发送了 400 条消息,MDB 处理了 700 条消息,其中 300 条是重复的。

看起来 MDB 重新连接工作正常,但如果托管“活动”MDB 的节点出现故障,消息可能会重复。

这可能是 weblogic JMS 实现的一个错误或一个功能。

I have a requirement that a single JMS message sent by a client must be delivered reliably (exactly-once) to two systems. These 2 systems are not HA-enabled, so the best suggestion that I came up with is to:

  1. create single queue where client posts to

  2. set up two "intermediate" queues

  3. use a custom "DuplicatorMDB" that will read messages from the client queue and post them to two queues within the same transaction.

client->JMSDQ->DuplicatorMDB->Q1->MDB->System1
                           \->Q2->MDB->System2

Is there any existing functionality like that? What would be the proper way to balance the system to keep it stable if one or both of the backend systems are down?

The application server is WebLogic 10.

I can't use topics for this because in a cluster topics will cause too much message duplication. If we have 2 instances, then with topics it'll go like this:

 
client->Topic-->MDB1@server1->System1
           | \->MDB2@server1->System2 
           \---->MDB1@server2->System1 
            \--->MDB2@server2->System2 

Thus every message will be delivered twice to System1 and twice to System2 and if there'll be 8 servers in a cluster, each message will be delivered 8 times. This is what I'd really like to avoid...

Finally I got some time to test it and here is what I observed:
2 nodes in a cluster. 2 JMS servers: jms1 on node1, jms2 on node2.
Distributed topic dt. MDB with durable subscription and jms-client-id=durableSubscriber. Started the system: 0 messages, mdb@node1 is up, mdb@node2 trying to connect periodically, but it can't because "Client id, durableSubscriber, is in use". As expected.

Sent in 100 messages:
jms1@dt messages current = 0, messages total = 100, consumers current = 1
I can see that node1 processed 100 messages.
jms2@dt messages current = 100, messages total = 100 , consumers current = 1
i.e. "duplicate" messages are pending in the topic.

Sent in another 100 messages, 100 were processed on the node1, 200 pending on node2.

Rebooted node1, mdb@node2 reconnected to dt and started processing "pending" messages. 200 messages were processed on node2.

After node1 is up, mdb@node1 can't connect to the dt, while mdb@node2 is connected.

jms1@dt messages current=0, messages total = 0, consumers current = 0
jms2@dt messages current=0, messages total = 200, consumers current = 1

Send in 100 more messages, I see that all 100 messages are processed on node2 and discarded on node1.

jms1@dt messages current=0, messages total = 100, consumers current = 0
jms2@dt messages current=0, messages total = 300, consumers current = 1

Now I reboot node2, mdb@node1 reconnects to dt. After reboot mdb@node2 reconnects to dt and mdb@node1 gets disconnected from dt.

jms1@dt messages current=0, messages total = 100, consumers current = 1
jms2@dt messages current=0, messages total = 0, consumers current = 1

I send in 100 messages, all are processed on node2 and stored in the topic on node1:

jms1@dt messages current=100, messages total = 200, consumers current = 1
jms2@dt messages current=0, messages total = 0, consumers current = 1

Then I shut down node2 and I see 100 "pending messages" being processed on node1 after mdb@node1 reconnects to the topic.

So the result is:
I sent 400 messages, 700 were processed by MDB out of which 300 were duplicates.

It looks like the MDB reconnection works good as expected, but the messages may be duplicated if a node hosting the "active" MDB goes down.

This might be a bug or a feature of weblogic JMS implementation.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

梦毁影碎の 2024-08-15 05:58:58

我没有使用过Weblogic,但是大多数JMS解决方案都有队列和主题的概念。您需要一个 JMS 主题。订阅者注册,主题确保消息被传递给每个订阅者一次。

配置详细信息

更新:如果您在集群环境中遇到问题,我会确保一切配置正确(这里是JMS 主题集群)。 Weblogic 在集群时会失败得如此惨烈,这听起来确实很奇怪。如果这不起作用,您可以查看第 3 方消息队列,例如支持 JMS 的 RabbitMQ并且绝对不会出现这个问题。

I haven't used Weblogic, but most JMS solutions have the concept of Queues and Topics. You want a JMS Topic. Subscribers register and the topic ensures that the message is delivered to each subscriber once.

Configuration details.

Update: If you are running into issues in a clustered environment, I would make sure that everything is configured properly (here is a guide for JMS Topic Clustering). It definitely sounds strange that Weblogic would fail so miserably when clustering. If that does not work, you could look into a 3rd party Messaging Queue, such as RabbitMQ, which support JMS and will definitely not have this issue.

桃扇骨 2024-08-15 05:58:58

这是 ESB 实现应该启用的行为类型。就处理开销而言,不会有太大差异,但在“管道”和应用程序代码之间分离关注点可能很有用。

碰巧,WebSphere JMS 实现支持安装满足此类需求的中介。我不知道 WebLogic 是否有类似的功能,或者它们相关的 ESB 产品是否适合您,但我建议您研究一下这些功能。您目前有一个简单的要求,并且您的代码肯定足够了,但是很容易想象一些小的附加要求(我们是否可以在传输到该目的地之前将此字段从美元转换为英镑,我们可以不发送消息吗?将此内容发送到该目的地...)瞧!您发现自己在编写自己的 ESB。

This is the kind of behaviour that an ESB implementation should enbale. In terms of processing overhead there would be no great difference, but it can be useful to have separation of concerns between "plumbing" and application code.

As it happens, the WebSphere JMS implementaion has support for installing mediations that address such requirements. I don't know whether WebLogic has something similar, or whether their associated ESB products are an option for you, but I would recommend investigating those capabilities. You currently have a simple requirement, and your code is surely sufficient, however it's quite easy to imagine how a few minor additional requirements (could we just convert this field from dollars to pounds before we transmit to that destination, could we not send messages with this content to that destination ...) and lo! you find yourself writing your own ESB.

蓝海 2024-08-15 05:58:58

[...] 因此,每条消息将被传送到 System1 两次,到 System2 两次,如果集群中有 8 个服务器,则每条消息将被传送 8 次。这是我真正想避免的......

这适用于非持久订阅,但不适用于持久订阅。为了持久化,所有 MDB 共享相同的连接 ID 和订阅 ID(默认情况下基于 MDB 名称),因此一次只有一个 MDB 能够附加和接收消息。第一个尝试的 MDB 将成功连接,其他 MDB 将检测到冲突并失败,但继续重试。因此,使用持久的主题订阅应该可以解决问题。

[...] Thus every message will be delivered twice to System1 and twice to System2 and if there'll be 8 servers in a cluster, each message will be delivered 8 times. This is what I'd really like to avoid...

This is right for non-durable subscriptions, but not for durable. For durable, all MDBs share the same connection-id and subscription-id (based on the MDB name by default), so only one MDB will be able to attach and receive messages at a time. The first MDB to try will succeed in connecting, the others will detect a conflict and fail, but keep retrying. So using a durable topic subscription should do the trick.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文