持久消息传递/服务总线 - 自行开发还是冒着学习曲线的风险?

发布于 2024-07-20 07:50:49 字数 707 浏览 6 评论 0原文

我们有一个客户端应用程序,需要向服务器发送消息以获取各种通知。 为了使客户端可以偶尔连接运行,我将采用消息队列方法。 队列处理将从队列中取出消息并调用 Web 服务,该服务将它们放入另一个队列中以最终进行处理。 这个问题是关于客户端环境的; 服务器环境已经确定。

我不想使用 MSMQ,因为我们无法控制所有客户端 PC 来正确安装/配置和保护 MSMQ,而且由于用于调查 MSMQ 队列内容的工具质量,支持更具挑战性。 SQL Server 2005 Express 位于所有计算机上,用于存储我们的应用程序的数据。

我目前有两个选择:

  1. 编写一个相当基本的持久消息队列,在序列化消息后将消息存储在表中,然后使用 ThreadPool.QueueUserWorkItem 让针对每条消息配置的处理程序处理它们类型。 所有这些都在 System.Transactions.TransactionScope 中,因此只有在成功处理后才会从持久队列中删除它们。
  2. 在客户端上使用 NServiceBus(这是我们作为一个团队使用的服务总线,因此 MassTransit 等不是选项),以及使用本地数据库的 Service Broker 传输。

我对服务总线的经验很少(我仍然没有真正了解服务总线术语),因此与编写更简单的东西以我需要的方式满足我的要求相比,我担心学习曲线(部署是一个考虑)。

有人有想法吗?

We have a client application that needs to send messages to a server for various notifications. In order that the client can run occasionally connected I'm going to go with a message-queue approach. The queue processing will take messages off the queue and call a web service that will put them on another queue to finally be processed. This question is about the client environment; the server environment is already decided on.

I don't want to use MSMQ because we don't have control over all the client PCs in order to install/configure and secure MSMQ properly, and because support is more challenging due to the quality of tooling for investigating the contents of MSMQ queues. SQL Server 2005 Express is on all the machines, and is used to store data for our application.

I currently have it down to two options:

  1. Write a fairly basic persistent message-queue that stores the messages in a table after serialising them, then uses ThreadPool.QueueUserWorkItem to have them processed by handlers configured against each message type. All in a System.Transactions.TransactionScope so they are only removed from the persistent queue if they are successfully processed.
  2. Use NServiceBus (this is the service bus we've gone with as a team, so MassTransit etc aren't options) on the client, with a Service Broker transport that uses the local database.

I have little experience with service buses (I still don't really get service bus terminology) so I'm concerned about the learning curve compared to writing something much more simple that meets my requirements in the way I need it to (deployment is a big consideration).

Does anyone have any thoughts?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

春夜浅 2024-07-27 07:50:49

好吧,我不知道我是否会建议使用 MSMQ,但我会建议“自己动手”需要考虑很多边缘情况。

即使使用线程池方法,请注意,如果您关心的话,可能会存在顺序问题 - 由于线程池处理工作项的方式,顺序发布到线程池的两个项目可能无法按顺序执行。

您还需要考虑消息的持久性,以及它们应该存在多长时间,如何检测“致命”未送达状态,以及在这种情况下该怎么做。

在您的应用程序在对消息进行排队的同时发生故障的情况下,还存在许多潜在的边缘情况 - 例如,它可能无法得到消息已排队的确认,即使它已排队。 是的,你可以确认队列确认,但你可以无休止地进入确认循环......

为什么不让应用程序检测它何时连接,并在此时发送所有数据?

Well, I don't know that I'm going to suggest MSMQ, but I will suggest that there's a lot of edge cases to think about for 'roll your own'.

Even with a thread pool approach, be aware that there may be ordering issues if you care - two items posted sequentially to thread pools may not be executed in order, due to how the thread pools handle work items.

You'll also need to think about persistence of messages, and how long they should exist, how to detect 'fatal' non-delivery status, and what to do in that case.

There's also a number of potential edge cases in scenarios where your app goes down around the same time as it's queueing a message - for instance, it may not get the acknowledgement that the message was queued, even though it was. Yes, you can ack the queue acknowledgement, but you can endlessly get into ack circles...

Why not just have the app detect when it gets connected, and send all the data at that point?

荆棘i 2024-07-27 07:50:49

在编写了我们自己的服务总线之后,我可以告诉您这不是一项微不足道的任务,您将遇到所有其他实现者已经遇到的相同边缘情况。 kyoryu 提出了两个非常重要的问题。

您是否自行部署还取决于您内部拥有的技能以及将来维护解决方案的技能。

另一个考虑因素是系统的最终规模及其可靠性要求。 内部解决方案是否能够充分扩展?

我们的点对点消息总线 Zebus 基于 ZeroMq(传输)、Cassandra(对等发现和持久性)和 Protobuf(序列化)。

  1. 没有客户端部署,因为在客户端上它只是一个库
  2. 消息持久性是使用 Cassandra 提供的以及处理许多不同的边缘情况(这在我们的生产环境中定期进行测试)
  3. 它性能良好,并且由于它是无代理解决方案,因此不存在单一性能瓶颈
  4. 不存在单点故障,因为目录可以与可用数据一起使用商店如 Cassandra

它是开源的并且经过生产测试 https://github.com/Abc-Arbitrage/Zebus< /a>

Having written our own service bus I can tell you it's not a trivial undertaking and you will run into the same edge cases that all other implementers have already run into. kyoryu brings up two very important ones.

Whether you roll your own also depends on the skills you have in-house and will have in the future to maintain the solution.

Another consideration is the eventual scale of the system and it's reliability requirements. Will the in-house solution scale sufficiently?

Our peer-to-peer message bus, Zebus, based on ZeroMq (transport), Cassandra (peer discovery and persistence) and Protobuf (serialisation).

  1. No client deployment as on the client it's just a library
  2. Message persistence is provided using Cassandra as well and handles many different edge cases (this is regularly tested in our production environment)
  3. It performs well and as it's a brokerless solution there is no single performance bottleneck
  4. There are no single points of failure as the Directory can be used with an available data store such as Cassandra

It is open source and production tested https://github.com/Abc-Arbitrage/Zebus

∝单色的世界 2024-07-27 07:50:49

最后,我编写了一个使用自己的 SqlTransport 配置的基本消息总线,以便在引发事件并通知单独的线程处理消息之前,将消息序列化并保存到 SQL Server 数据库表中。

In the end I wrote a basic messagebus configured with my own SqlTransport so that messages are serialised and saved to a SQL Server database table, before events are raised and a separate thread is signalled to process the messages.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文