多个消费者&连接到消息队列的生产者,这在 AMQP 中可能吗?
我想创建一个能够 OCR 文本的进程群。 我考虑过使用由多个 OCR 进程读取的单个消息队列。
我想确保:
- 队列中的每条消息最终都会得到处理
- 工作或多或少均匀分布
- 图像将仅由一个 OCR 进程解析
- 一个 OCR 进程不会一次获取多条消息(因此任何其他免费 OCR进程可以处理该消息)。
使用 AMQP 可以做到吗?
我打算使用 python 和rabbitmq
I'd like to create a farm of processes that are able to OCR text.
I've thought about using a single queue of messages which is read by multiple OCR processes.
I would like to ensure that:
- each message in queue is eventually processed
- the work is more or less equally distributed
- an image will be parsed only by one OCR process
- An OCR process won't get multiple messages at once (so that any other free OCR process can handle the message).
Is that possible to do using AMQP?
I'm planning to use python and rabbitmq
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
是的,正如@nailxx 指出的那样。 AMQP 编程模型与 JMS 略有不同,因为您只有队列,队列可以在工作程序之间共享,或者由单个工作程序私有使用。您还可以轻松设置 RabbitMQ 来执行 PubSub 用例或 JMS 中所谓的主题。请访问 入门 页面。 rabbitmq.com/" rel="nofollow noreferrer">RabbitMQ 网站 查找大量有关此内容的有用信息。
现在,特别是对于您的用例,已经有很多可用的工具。 Celery 是人们经常使用的一种。这是一篇关于它的博客文章,我认为这会帮助您开始:
如果您有任何疑问,请给我们发送电子邮件或发布到rabbitmq-discuss 邮件列表。
Yes, as @nailxx points out. The AMQP programming model is slightly different from JMS in that you only have queues, which can be shared between workers, or used privately by a single worker. You can also easily set up RabbitMQ to do PubSub use cases or what in JMS are called topics. Please go to our Getting Started page on the RabbitMQ web site to find a ton of helpful info about this.
Now, for your use case in particular, there are already plenty of tools available. One that people are using a lot, and that is well supported, is Celery. Here is a blog post about it, that I think will help you get started:
If you have any questions please email us or post to the rabbitmq-discuss mailing list.
是的,这是可能的。我正在开发的实时MMO游戏的服务器集群就是这样操作的。我们使用 ActiveMQ,但我认为这一切也可以通过 RabbitMQ 实现。
除了最后一件之外,您提到的所有物品都可以开箱即用。
为了使最后一个消息以所需的方式工作,消费者在订阅队列时发送特定于 AMQ 的参数:
此设置保证消费者在接收一条消息后不会再接收任何消息,直到它发送 < code>ack 到 AMQ。我相信 RabbitMQ 中也存在类似的东西。
Yes, that's possible. Server cluster for a real-time MMO game I'm working on operate this way. We use ActiveMQ, but I think all this possible with RabbitMQ as well.
All items that you mentioned you get out of the box, except last one.
To make last one work in desired way, consumers send AMQ-specific argument when subscribing to the queue:
This setting guarantees that consumer will not take any more messages after it took one and until it send an
ack
to AMQ. I believe something similar exists in RabbitMQ.