在这种情况下,JMS 主题是否足够?或者我应该去别处看看?

发布于 2024-12-05 18:29:24 字数 711 浏览 0 评论 0原文

有一个控制实体和多个“工作”实体。控制实体向工作实体请求某些数据,它们将以自己的方式获取并返回这些数据。

由于控制实体可以不知道工作实体(并且可以在任何时候添加/删除工作实体),因此在它们之间放置 JMS 提供程序听起来是个好主意。至少这是假设。

由于它是一对多关系(控制器 -> 工作人员),因此 JMS 主题将是正确的解决方案。但是,由于控制实体取决于工作人员的返回值,因此请求/回复功能也很好(在某个地方,我读到了有关 TopicRequester 的内容,但我似乎找不到有效的示例) 。请求/回复是典型的队列功能。

为了尝试以请求/回复的方式使用主题,我创建了两个 JMS 主题:requestresponse。控制器发布到request主题并订阅response主题。每个工作人员都订阅request主题并发布到response主题。为了匹配请求和响应,控制器将使用过滤器(使用会话 ID 作为值)订阅对 response 主题的每个请求。工作人员发布到 response 主题的消息具有与其关联的会话 ID。

现在,这感觉不像是一个解决方案(相反,它使用 JMS 作为锤子,并将问题(以及更多问题)视为钉子)。在这种情况下,JMS 到底是一个解决方案吗?或者还有其他我忽略的解决方案吗?

There is one controlling entity and several 'worker' entities. The controlling entity requests certain data from the worker entities, which they will fetch and return in their own manner.

Since the controlling entity can agnostic about the worker entities (and the working entities can be added/removed at any point), putting a JMS provider in between them sounds like a good idea. That's the assumption at least.

Since it is an one-to-many relation (controller -> workers), a JMS Topic would be the right solution. But, since the controlling entity is depending on the return values of the workers, request/reply functionality would be nice as well (somewhere, I read about the TopicRequester but I cannot seem to find a working example). Request/reply is typical Queue functionality.

As an attempt to use topics in a request/reply sort-of-way, I created two JMS topis: request and response. The controller publishes to the request topic and is subscribed to the response topic. Every worker is subscribed to the request topic and publishes to the response topic. To match requests and responses the controller will subscribe for each request to the response topic with a filter (using a session id as the value). The messages workers publish to the response topic have the session id associated with them.

Now this does not feel like a solution (rather it uses JMS as a hammer and treats the problem (and some more) as a nail). Is JMS in this situation a solution at all? Or are there other solutions I'm overlooking?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

不顾 2024-12-12 18:29:24

你的方法对我来说有点道理。我认为消息系统可以发挥作用。我认为使用主题是错误的。查看 企业服务总线 的 wiki 页面。它比您需要的要复杂一些,但是您的用例的基本思想是您有一个工作人员能够从一个队列中读取数据,进行一些处理并将处理后的数据添加回另一个队列。

一个主题的问题在于,所有工作人员都会同时收到消息,并且他们都会独立处理该消息。听起来您一次只需要一名工作人员处理每个请求。我认为您将其作为主题,以便不同类型的工作人员也可以侦听同一队列并仅响应某些请求。为此,您最好为每种类型的工作创建一个新队列。您可能会将它们成对出现,因此您有一个 work_a_request 队列和 work_a_response 队列。或者,如果您的控制器能够从数据中找出响应类型,它们都可以写入单个响应队列。

如果您还没有选择消息队列供应商,我会推荐 RabbitMQ 因为它很容易设置,易于添加新队列(尤其是动态地),并且具有非常好的 Spring 支持(尽管大多数主要消息系统都有 Spring 支持,并且您甚至可能没有使用 Spring)。

我也不确定你要完成什么过滤器。如果您确保发送给工作人员的消息包含完成工作所需的所有信息,并且返回的响应消息包含控制器完成处理所需的所有信息,我认为您不需要它们。

Your approach sort of makes sense to me. I think a messaging system could work. I think using topics are wrong. Take a look at the wiki page for Enterprise Service Bus. It's a little more complicated than you need, but the basic idea for your use case, is that you have a worker that is capable of reading from one queue, doing some processing and adding the processed data back to another queue.

The problem with a topic is that all workers will get the message at the same time and they will all work on it independently. It sounds like you only want one worker at a time working on each request. I think you have it as a topic so different types of workers can also listen to the same queue and only respond to certain requests. For that, you are better off just creating a new queue for each type of work. You could potentially have them in pairs, so you have a work_a_request queue and work_a_response queue. Or if your controller is capable of figuring out the type of response from the data, they can all write to a single response queue.

If you haven't chosen an Message Queue vendor yet, I would recommend RabbitMQ as it's easy to set-up, easy to add new queues (especially dynamically) and has really good spring support (although most major messaging systems have spring support and you may not even be using spring).

I'm also not sure what you are accomplishing the filters. If you ensure the messages to the workers contain all the information needed to do the work and the response messages back contain all the information your controller needs to finish the processing, I don't think you need them.

梦里的微风 2024-12-12 18:29:24

我会简单地使用两个 JMS 队列。

第一个是所有请求都进行的那个。工作人员将监听队列,并在自己的时间、以自己的方式处理它们。

完成后,他们会将请求与响应捆绑在一起,并将其放入另一个队列中以供最终进程处理。这样,提交过程就不需要保留请求,它们只需遵循整个过程即可。最后一个进程将侦听第二个队列,并适当地处理请求/响应对。

如果不需要消息可靠,或者不需要实际进程跨越 JVM 或机器,那么这一切都可以通过单个进程和标准 java 线程(例如 BlockingQueues 和 ExecutorServices)来完成。

如果需要累积相关响应,那么您需要捕获任何必要的分组数据,并让 Queue 2 侦听进程累积结果。或者您可以将结果保存在数据库中。

例如,如果您知道工作集有五个元素,则可以使用该信息对请求进行排队(1 of 5、2 of 5 等)。当每个过程完成时,最后一个过程可以更新数据库,对元素进行计数。当它看到所有部分都已完成(以任何顺序)时,它将结果标记为完成。稍后,您将进行一些审核过程扫描,以查找在一段时间内未完成的未完成作业(可能是其中一条消息出错),以便您可以更好地处理它们。或者原始处理器可以将请求写入单独的“这个出错了”队列以进行缓解和重新提交。

如果您将 JMS 与事务一起使用,如果其中一个处理器发生故障,事务将回滚,并且消息将保留在队列中以供其中一个幸存的处理器处理,因此这是 JMS 的另一个优点。

这种处理的技巧是尝试用消息推送状态,或者将其外部化并发送对状态的引用,从而使每个组件有效地无状态。这有助于扩展和可靠性,因为任何组件都可能发生故障(当然,除了灾难性的 JMS 故障),并且当您解决问题并重新启动它们时,只需从上次停下的地方继续。

如果您处于请求/响应模式(例如需要响应的 servlet),您可以使用 Servlet 3.0 异步 servlet 轻松地将事情搁置,或者您可以将本地对象放在内部映射上,并使用 Something 键控例如会话 ID,然后您在该键中使用 Object.wait() 。然后,您的队列 2 侦听器将获取响应,完成处理,然后使用会话 ID(与消息一起发送并在整个管道中保留)来查找
您正在等待的对象,然后它可以简单地使用 Object.notify() 来告诉 servlet 继续。

是的,这会在等待时将一个线程粘在 servlet 容器中,这就是为什么新的异步东西更好,但你要根据自己的手来工作。您还可以向 Object.wait() 添加超时,如果超时,则处理时间会很长,因此您可以优雅地提醒客户端。

这基本上使您摆脱了过滤器等以及回复队列等。设置起来非常简单。

I would simply use two JMS queues.

The first one is the one that all of the requests go on. The workers will listen to the queue, and process them in their own time, in their own way.

Once complete, they will put bundle the request with the response and put that on another queue for the final process to handle. This way there's no need for the the submitting process to retain the requests, they just follow along with the entire procedure. A final process will listen to the second queue, and handle the request/response pairs appropriately.

If there's no need for the message to be reliable, or if there's no need for the actual processes to span JVMs or machines, then this can all be done with a single process and standard java threading (such as BlockingQueues and ExecutorServices).

If there's a need to accumulate related responses, then you'll need to capture whatever grouping data is necessary and have the Queue 2 listening process accumulate results. Or you can persist the results in a database.

For example, if you know your working set has five elements, you can queue up the requests with that information (1 of 5, 2 of 5, etc.). As each one finishes, the final process can update the database, counting elements. When it sees all of the pieces have been completed (in any order), it marks the result as complete. Later you would have some audit process scan for incomplete jobs that have not finished within some time (perhaps one of the messages erred out), so you can handle them better. Or the original processors can write the request to a separate "this one went bad" queue for mitigation and resubmission.

If you use JMS with transaction, if one of the processors fails, the transaction will roll back and the message will be retained on the queue for processing by one of the surviving processors, so that's another advantage of JMS.

The trick with this kind of processing is to try and push the state with message, or externalize it and send references to the state, thus making each component effectively stateless. This aids scaling and reliability since any component can fail (besides catastrophic JMS failure, naturally), and just pick up where you left off when you get the problem resolved an get them restarted.

If you're in a request/response mode (such as a servlet needing to respond), you can use Servlet 3.0 Async servlets to easily put things on hold, or you can put a local object on a internal map, keyed with the something such as the Session ID, then you Object.wait() in that key. Then, your Queue 2 listener will get the response, finalize the processing, and then use the Session ID (sent with message and retained through out the pipeline) to look up
the object that you're waiting on, then it can simply Object.notify() it to tell the servlet to continue.

Yes, this sticks a thread in the servlet container while waiting, that's why the new async stuff is better, but you work with the hand you're dealt. You can also add a timeout to the Object.wait(), if it times out, the processing took to long so you can gracefully alert the client.

This basically frees you from filters and such, and reply queues, etc. It's pretty simple to set it all up.

多彩岁月 2024-12-12 18:29:24

实际答案应该取决于您的工作人员实体是否是外部各方、物理位置位于外部网络、工作人员实体完成工作的预期时间等。但是您要解决的问题是一对多通信...u 添加系统中的 jms 协议只是因为您希望所有实体都能够使用 jms 协议或异步进行通信是原因...前一个原因没有意义...如果是后一个原因,您可以选择其他通信协议,例如 -方式Web服务调用。
您可以使用最新的 java 并发 API 来创建对不同工作实体的多线程异步单向 Web 服务调用...

Well actual answer should depend upon whether your worker entities are external parties, physical located outside network, time expected for worker entity to finish their work etc..but problem you are trying to solve is one-to-many communication...u added jms protocol in your system just because you want all entities to be able to talk in jms protocol or asynchronous is reason...former reason does not make sense...if it is latter reason, you can choose other communication protocol like one-way web service call.
You can use latest java concurrent APIs to create multi-threaded asynchronous one-way web service call to different worker entities...

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文