消息队列和服务总线的消息粒度
我正在开发一个应用程序，该应用程序可能会在客户端上相当紧密的循环中生成数千条消息，以便在服务器上进行处理。 事件链类似于：
- 客户端处理项目，放入本地队列。
- 本地队列处理拾取消息并调用 Web 服务。
- Web 服务在服务器上的服务总线中创建消息。
- 服务总线将消息处理到数据库。
这个想法是所有通信都是异步的，因为 Web 服务将有许多客户端。 我知道 MSMQ 可以直接执行此操作，但我们并不总是在客户端上具有这种管理功能来设置安全性等内容。
我的问题是关于每个阶段消息的粒度。 最简单的方法意味着在客户端上处理的每个项目都会生成一条客户端消息/Web 服务调用/服务总线消息。 这很好，但我知道如果可能的话，最好对 Web 服务调用进行批处理，除非需要在大粒度 Web 服务 DTO 与数据库上的短期运行事务之间进行权衡。 这种特殊的场景不需要“业务事务”，即处理所有项目或不处理任何项目，我只是希望在消息大小与 Web 服务调用数量与数据库事务之间实现最佳平衡。
有什么建议吗？
I'm working on an application that may generate thousands of messages in a fairly tight loop on a client, to be processed on a server. The chain of events is something like:
- Client processes item, places in local queue.
- Local queue processing picks up messages and calls web service.
- Web service creates message in service bus on server.
- Service bus processes message to database.
The idea being that all communications are asynchronous, as there will be many clients for the web service. I know that MSMQ can do this directly, but we don't always have that kind of admin capability on the clients to set things up like security etc.
My question is about the granularity of the messages at each stage. The simplest method would mean that each item processed on the client generates one client message/web service call/service bus message. That's fine, but I know it's better for the web service calls to be batched up if possible, except there's a tradeoff between large granularity web service DTOs, versus short-running transactions on the database. This particular scenario does not require a "business transaction", where all or none items are processed, I'm just looking to achieve the best balance of message size vs. number of web service calls vs. database transactions.
Any advice?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
喋喋不休的接口（即大量的消息）往往会产生很高的开销，因为将传入消息（以及客户端上的回复）分派给正确的代码来处理消息（这将是每条消息的固定成本） 。 而大消息往往会使用处理消息时的资源。
此外，正在进行的大量 Web 服务调用意味着需要管理大量 TCP/IP 连接，并且并发问题（包括锁定数据库）可能会成为问题。
但是，如果没有消息处理的一些细节，就很难具体，除了由于固定开销而针对聊天界面的一般建议。
Chatty interfaces (i.e. lots and lots of messages) will tend to have a high overhead from dispatching the incoming message (and, on the client, the reply) to the correct code to process the message (this will be a fixed cost per message). While big messages tend to use the resources in processing the message.
Additionally a lot of web service calls in progress will mean a lot of TCP/IP connections to manage, and concurrency issues (including locking in a database) might become an issue.
But without some details of the processing of the message it is hard to be specific, other than the general advice against chatty interfaces because of the fixed overheads.
先测量，后优化。 除非您可以进行粗略估计，表明最简单的解决方案会产生不可接受的高负载，否则请尝试一下，建立良好的监督测量，看看它的性能和扩展情况如何。 然后开始考虑批处理的数量和地点。
当然，这种方法要求您能够在部署后更改 Web 服务接口，因此您需要一种版本控制方法来处理可能尚未重新设计的客户端，并行支持多个 WS 版本。 但无论如何，不考虑版本控制几乎总是会让你陷入次优的界面中。
Measure first, optimize later. Unless you can make a back-of-the-envelope estimate that shows that the simplest solution yields unacceptably high loads, try it, establish good supervisory measurements, see how it performs and scales. Then start thinking about how much to batch and where.
This approach, of course, requires you to be able to change the web service interface after deployment, so you need a versioning approach to deal with clients which may not have been redesigned, supporting several WS versions in parallel. But not thinking about versioning almost always traps you in suboptimal interfaces, anyway.
抽象消息队列
并拥有可交换的消息队列后端。 通过这种方式，您可以测试许多后端，并在您选择错误的后端或逐渐喜欢出现的新后端时为自己提供轻松的救助。 消息传递的开销通常是打包和处理请求。 不同的系统是针对不同级别的流量和随时间变化的不同对称性而设计的。
如果您抽象出基本功能，则可以根据需求的变化或更准确地评估来交换机制。
您还可以在接收者压力发生变化时，在应用程序或消息路由的不同部分转换来自不同队列类型的消息，因为它们正在处理，例如 1000:1/s 与更高级别的 10:1/s。
祝你好运
Abstract the message queue
and have a swappable message queue backend. This way you can test many backends and give yourself an easy bail-out should you pick the wrong one or grow to like a new one that appears. The overhead of messaging is usually packing and handling the request. Different systems are designed for different levels traffic and different symmetries over time.
If you abstract out the basic features you can swap the mechanics in and out as your needs change, or are more accurately assessed.
You can also translate messages from differing queue types at various portions of the application or message route as the recipient's stresses change because they are handling, for example 1000:1/s vs 10:1/s on a higher level.
Good Luck