RabbitMQ 中的作业依赖关系
我正在尝试找出如何最好地设置以下场景:
- 将多个 A 类型作业添加到队列中
- 当所有 A 类型作业完成时,将需要 B 或 C 类型作业(每个 A 类型作业一个) )
- 当所有 A、B 和 C 类型作业完成后,将需要最终的 D 类型作业
因此基本上我们对队列中的作业有一些依赖性,这样我们就不想开始运行需要其他作业的作业完全的。有建立这样一个系统的指南吗? A类工作完成后是否应该添加B类或C类工作?是否应该预先添加所有工作并以某种方式告诉工人在准备好之前不要取消它们?
如果我必须手动管理这种依赖关系,这两种方法都有优点和缺点,但我很好奇是否可以使用不同的模式来代替它,它可能会以更简单的方式完成相同的事情。
I am trying to figure out how to best setup the following scenario:
- Multiple A-type jobs are added into the queue
- When all A-type jobs are completed, a B or C-type job will be needed (one per A-type job)
- When all A, B and C type jobs are completed, a final D-type job will be needed
So basically we have some dependencies on jobs in the queue such that we don't want to start running jobs that require other jobs to be completed. Is there a guideline for setting up such a system? Should A-type jobs add B or C type jobs after their work is completed? Should all jobs be added up front and somehow tell the workers not to pull them until they are ready?
There are pros and cons of both approaches if I have to manually manage this dependency but I am curious if there is a different pattern I could use instead that might accomplish the same thing but in an easier way.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我想知道 routing slip 模式是否可以在这里启发您,即发送原始的 A 消息定义下一个消息(B,C)应该是什么(在运行时应用什么规则来计算它)的“slip”。
我还会在单据中添加唯一的相关 ID 和相关组大小(在这种情况下:A 消息的总数)。
对于最终的 D 作业,我会让 B/C 流程步骤发送“D-ready”请求消息。当所有 D 就绪消息都已收集完毕(基于相关 ID/组大小)时,我将触发最终的 D 作业。
I'm wondering if the routing slip pattern could inspire you here, ie send in the original A messages a "slip" that defines what the next messages (B, C) should be (what rules to apply to figure it out at runtime).
I would also add in the slip a unique correlation ID and the correlation group size (in that case: the total number of A messages).
For the final D job, I'd have the B/C process steps send a "D-ready" request message. When all the D-ready messages would have been collected (based on correlation ID/group size), I would trigger the final D job.