使用 ESB（WSO2、Mule、Petals、Service Mix）进行二进制文件处理
我热衷于在新项目中使用 ESB（可能是 WSO2、Mule、Petals 或 Service Mix）。
我有许多在 ESB 中运行良好的业务流程/工作流程：但是我在一个特定的场景中遇到了困难。
基本要求是通过多个阶段处理传入的二进制文件：
- 使用外部工具预处理文件；
- 根据一些配置数据分割文件；
- 处理每个较小的文件以计算“分数”。其中一些将由我编写的代码处理，但在某些情况下也会调用外部软件；
- 将所有计算出的分数加载到数据库中。这必须在流程结束时完成，而不是增量完成（因为只有在处理文件期间才能找到主键）；
- 将初始文件移动到存档区域。
传入的文件将被放入本地目录中。所有工具都支持目录轮询，因此这不是问题。
然而，根据我的阅读，大多数工具都希望消息的有效负载易于读取，以便可以直接在有效负载上执行处理。就我而言，我只想使用传入文件来生成我的真实有效负载数据：我将写入数据库的值。
这是 ESB 可以支持的模式吗？有没有关于使用上述工具进行此类数据处理的文章？我应该考虑其他技术/模式吗？
希望这是有道理的！
非常感谢，
肯
I'm keen to use an ESB (probably WSO2, Mule, Petals or Service Mix) on a new project.
I have a number of business processes/ workflows which will work well in an ESB: however I'm struggling with one particular scenario.
The basic requirement is to process incoming binary files through a number of stages:
- pre-processing the file using an external tool;
- splitting the file based on some configuration data;
- processing each of these smaller files to calculate a "score". Some of this will be handled by code I will write, but external software will also be called in some instances;
- loading all the calculated scores into a database. This must be done at the end of the flow rather than incrementally (as the primary key will be only be found during processing of the file);
- moving the initial file to an archive area.
The incoming files will be dropped into a local directory. All the tools support polling of a directory so this isn't a problem.
However, from what I've read most of the tools expect the payload of the message to be easily readable so that processing can be performed directly on the payload. In my case I merely want to use the incoming file to generate my real payload data: the values which I will write to the database.
Is this a pattern that ESBs can support? Are there any articles about this kind of data processing with any of the above tools? Should I be looking at any other technology/patterns?
Hope this makes sense!
Many thanks,
Ken
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
WSO2 提供侦听文件系统中的文件的功能。但是，根据您的查询，您似乎需要使用专有格式处理二进制文件。如果是这样，您可以使用 WSO2 ESB vfs 传输 [1]，并且必须实现自定义消息格式化程序和构建器（[2] 中描述的简单 java 代码）
[1] http://wso2.org/project/esb/java/4.0.0/docs/samples/transport_samples.html#Sample254
[2] http://charithwiki.blogspot。 com/2010/11/how-to-write-axis2-message-builder.html
WSO2 provides capability to listen to the files in the file system. However, as per your query it seems you need to process a binary file with your proprietary format. If so, you may use WSO2 ESB vfs transport[1] and you have to implement a custom message formatter and a builder (trivial java code as described on [2])
[1] http://wso2.org/project/esb/java/4.0.0/docs/samples/transport_samples.html#Sample254
[2] http://charithwiki.blogspot.com/2010/11/how-to-write-axis2-message-builder.html
这应该可以通过 ESB 实现。特别是使用 UltraESB，您可以通过其内置文件传输来实现此目的，默认情况下，它支持在处理后将文件移动到存档区域。
UltraESB 的拆分和聚合功能可用于将文件拆分为多个部分并单独处理它们。 “分数”生成可以使用 Java 或任何其他脚本语言来完成，这些语言利用 UltraESB API 与消息（在本例中为文件的一部分）及其环境进行交互。
借助 UltraESB 的数据库连接机制，可以轻松完成数据库交互。此外，您可以使用 UltraESB 以事务方式执行这些更新。
如果文件是标准的 CSV 或 EDI 格式，那么您可以使用 csv 或 EDI 支持轻松处理该文件。所有这些都可以通过 Java 或任何脚本语言进行极其简单的配置。
[1] - http:// /docs.adroitlogic.org/display/esb/Transactional+ESB+use+cases+made+simple+with+the+UltraESB
[2] - http://docs.adroitlogic.org
This should be possible with an ESB. In particular with UltraESB you could do this with its built-in file transport, which supports the moving of the file after processing to an archive area by default.
The Splitting and aggregation features of the UltraESB could be used to split the file into parts and process them separately. The "score" generation can be done with Java or any other scripting language which leverage the UltraESB API to interact with the message (in this case the part of the file) and its environment.
The DB interactions can be easily done with the UltraESB with its data base connection mechanisms. Further, you could do these updates transactionally with UltraESB.
If the file is of a standard CSV or EDI format, then you can use the csv or EDI support to process the file easily. All these via an extremely easy configuration with either Java or any scripting language.
[1] - http://docs.adroitlogic.org/display/esb/Transactional+ESB+use+cases+made+simple+with+the+UltraESB
[2] - http://docs.adroitlogic.org