分布式系统中的消息传递和信令

发布于 2024-08-18 01:21:59 字数 547 浏览 9 评论 0原文

我有一个分布式视频分析系统,其组成为:

1. feature extraction: generated lots of features(20+) from each frame of the video
2. multiple detectors(in different machine): 
  * Each of them will get a subset of feature
  * Each of them needs the features from multiple frames. 
  * Eg. Detector 1 needs feature 1-5 from 3 frames to start processing; Detector 2 needs feature 2-8 from 8 frames to start processing

我的问题是:如何在特征提取块和多个检测器之间进行通信,最好是实时通信?我一直在研究一种事件总线,但它仅适用于一个进程,Hadoop 中的 ZooKeeper 会是更好的解决方案吗?

我正在使用Java。欢迎任何建议。

I have a distributed video analysis system, which is composed of:

1. feature extraction: generated lots of features(20+) from each frame of the video
2. multiple detectors(in different machine): 
  * Each of them will get a subset of feature
  * Each of them needs the features from multiple frames. 
  * Eg. Detector 1 needs feature 1-5 from 3 frames to start processing; Detector 2 needs feature 2-8 from 8 frames to start processing

My question is: how to do the communication between feature extraction block and multiple detectors, preferably in real-time? I have been looking at an event bus, but it's only for one process, will ZooKeeper in Hadoop be a better solution?

I am using Java. Any suggestion is welcomed.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

木格 2024-08-25 01:21:59

在 Java 中,您可以考虑使用 RabbitMQ 消息总线。有一个带有 Java 绑定 的库。

RabbitMQ 基于经过验证的
平台,提供异常高的
可靠性、可用性和
可扩展性以及良好的吞吐量
和延迟性能
可预测且一致。它有一个
紧凑、易于维护的代码库
允许快速定制和热
部署。有广泛的
管理、监控设施,
控制和调试,它是
得到全方位的支持
商业支持服务和
活跃的社区开发包
扩展核心系统。

In Java, you could consider using RabbitMQ message bus. There is a library with Java bindings.

RabbitMQ is based on a proven
platform, offering exceptionally high
reliability, availability and
scalability along with good throughput
and latency performance that is
predictable and consistent. It has a
compact, easily maintainable code base
allowing rapid customisation and hot
deployment. There are extensive
facilities for management, monitoring,
control and debugging and it is
supported by a full range of
commercial support services and an
active community developing packages
that extend the core system.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文