EAI/中间件：内联或引用的大量消息
假设中间件传输但不处理巨大的数据块，是否有通过中间件或 EAI 系统处理处理大量数据（MBytes？GBytes？）的消息或请求/响应的良好实践？
通过消息传输数据以保持一致性，但可能多次在系统之间复制大量数据
将数据保存在存储或数据存储库（文件系统、数据库等）中并传输对数据的引用，但会失去消息和数据之间的紧密耦合
此问题的常见解决方案是什么？ EAI 或中间件产品如何支持这一点？
Are there good practices for processing messages or requests/responses dealing with huge amounts of data (MBytes? GBytes?) via an middleware or an EAI system, assunming that the middleware transports but does not process the huge data blobs?
Transport the data with the message in order to keep coherence but probably several times copy huge amounts of data between systems
Keep the data in a store or data repository (file system, database,...) and transport a reference to the data, however loosing the close coupling between the message and the data
What is the common solution to this problem? How is this supported by EAI or middleware products?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
根据我的经验，限制消息的大小是很好的选择。在我所知道的环境（Websphere MQ、SonicMQ）中，我发现 5-10 MB 对于大多数消息来说是一个不错的值。在许多情况下，拥有很少且大小高达 100 MB 的消息不会有什么坏处 ->但在那里你必须考虑内存消耗，......在大多数消息传递API中，尤其是JMS，你至少会一次将消息完全存储在内存中，这可能会让你在开始并行处理时感到一些头痛。
我们使用两种方法作为解决方法：
通常，只要消息包含结构化数据，我们就会使用第一个解决方案，而在媒体分发的情况下则使用第二个解决方案。
对于第一个解决方案，我们也倾向于不使用提供的特定解决方案，而是在我们的自定义实现中实现此逻辑 - 因此我们不绑定到特定的消息传递系统。
In my experience it is good to limit the size of the messages. In the environments i know (Websphere MQ, SonicMQ) i found 5-10 MB a good value for most of the messages. In many cases it will not hurt to have very few messages with sizes up to 100 MB -> but there you will have to think about memory consumption, .... In most messaging APIs, especially JMS you will at least have the message completely in memory one time, which may cause you some headaches when starting parallel processing.
There are two things we use as a workaround:
Normally we use the first solution whenever the messages contain structured data, the second one is used in cases of media distribution.
For the first solution we do also tend not to use provides specific solutions, but implement this logic in our custom implementation - therefor we are not bound to a specific messaging system.
如果在源系统和目标系统之间建立直接连接是可行的，那么仅通过消息发送引用来启动传输将可行，但是，正如您所指出的，您会失去松散耦合。
根据《企业集成模式》一书，您可以实现消息序列模式。
根据我使用 TIBCO 等 EAI 系统的经验。如何处理任意大量的数据几乎取决于您。
If it is practical to make a direct connection between the source and destination system then sending only a reference through a message to start the transfer will work but, as you noted you lose the loose coupling.
According to the book Enterprise Integration Patterns you could implement the Message sequence pattern.
In my experience with EAI systems such as TIBCO. It pretty much up to you to figure our how to handle arbitrarily large amounts of data.