当您已经拥有 SQL Server 和 BizTalk 时使用 MSMQ

发布于 2024-12-27 01:22:33 字数 503 浏览 1 评论 0原文

简单的问题:是否有充分的理由将 MSMQ 添加到已有多个 BizTalk 和 SQL Server 节点的现有消息传递框架中?

背景是这样的:我们有一个消息传递框架来处理账单,目前负载相当低(每天最多 10,000 个),但它正在增加。我们使用 BizTalk 和 SQL Server 进行所有处理,并且在(同步)插入其中一个数据库(不是 BizTalk 消息框)时开始注意到一些超时。我们的一位资深程序员建议我们使用MSMQ来保存(异步)导致超时的数据并稍后处理;他设计的解决方案有效并且即将部署,但考虑到我们可以使用 BizTalk 本身或 SQL Server Service Broker (SSSB),我仍然想知道这是否是正确的决定。关于这三种技术有很多讨论,但通常都是关于必须选择其中一种而不是其他技术,我还没有看到任何人已经拥有 BizTalk 和 SSSB 并决定将 MSMQ 添加到组合中。在我们的例子中,我认为这是对我们的技术堆栈的不必要的补充,但这可能是我自己的偏见(也是无知),因为我更了解 SSSB 并且从未对 MSMQ 做过任何大事。你怎么认为?

Simple question: Is there any good reason to add MSMQ to an existing messaging framework which already has multiple BizTalk and SQL Server nodes?

Here's the background: We have a messaging framework to process bills, the load is rather low right now (at most 10,000 a day), but it's ramping up. We use BizTalk and SQL Server for all the processing, and we started noticing a few timeouts when inserting (synchronously) into one of the databases (NOT the BizTalk message box). One of our senior programmers suggested we use MSMQ to save (asynchronously) the data that causes the timeout and process it later; the solution he designed works and it's about to be deployed, but I'm still wondering if that was the right decision, considering that we could have used BizTalk itself or SQL Server Service Broker (SSSB). There's a lot of discussions about those three technologies, but they're usually about having to choose one of them over the others, I haven't seen any case of anyone who already had BizTalk and SSSB and decided to add MSMQ to the mix. In our case I think it's an unnecessary addition to our technology stack, but that may be my own bias (and ignorance too), since I know SSSB better and never did anything big with MSMQ. What do you think?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

讽刺将军 2025-01-03 01:22:33

听起来你应该弄清楚为什么你的插入花费了这么长时间,并解决它。对于运行 SQL Server 的体面机器来说,每天 10,000 个“不算什么”。

编辑:

添加任何类型的异步处理都是一种搁置的形式。假设您的插入需要一分钟(我意识到它们可能不需要,但为了争论)。如果您将插入设为异步,则每天仍然只能处理 1440 个插入,直到开始落后。最终你总是需要加快插入速度。

话虽如此,我认为在这种情况下使用 MSMQ 相对于 SSSB 并没有任何引人注目的好处(反之亦然)。可能有人会说,使用 MSMQ,您需要手动编写一个侦听器守护进程来执行插入操作,而使用 SSSB,您可以在数据库中自动完成该操作。另一方面,使用 MSMQ,您可以将消息存储卸载到另一台服务器,从而可能减轻 SQL Server 的一些直接压力。

It sounds like you should figure out why your inserts are taking so long, and fix that instead. 10,000 / day is nothing for a decent box running SQL Server.

EDIT:

Adding any sort of asynchronous processing is a form of kicking the can down the road. Assume your inserts take one minute (I realize they probably don't, but for argument's sake). If you make your inserts asynchronous, you can still only handle 1440 inserts per day until you start falling behind. You are always going to need to speed up your inserts eventually.

Now with that said, I don't think that there is any compelling benefit in this case of using MSMQ over SSSB (or vice-versa). It could be argued that with MSMQ you need to hand-code a listener daemon that does your inserts, whereas with SSSB you have that automatically within the database. On the other hand, with MSMQ you are offloading the storage of the messages to another server, potentially offloading some of the immediate stress from your SQL Server.

一江春梦 2025-01-03 01:22:33

我认为,如果您只想使数据库调用脱机,那么您可以使用 BizTalk 来实现(例如,通过创建“脱机”主机 - 从而创建新的主机队列)。

msmq 真正擅长的地方是 BizTalk 的入站方面。系统可以调用 BizTalk,而不关心 BizTalk 本身的可用性。这些消息将一直保留,直到 BizTalk 再次可用为止。

I would argue that if you just wanted to take the database calls off-line then you could do that with BizTalk (for example, by creating an "offline" host - thereby creating a new host queue).

Where msmq really excels is on the inbound side of BizTalk. Systems can call to BizTalk not caring about the availability of BizTalk itself. The messages will just hang around until BizTalk is available again.

沫离伤花 2025-01-03 01:22:33

我和 Hugh 一样,我们已经成功地将 MSMQ(和 IBM MQ 系列)与 BizTalk 结合使用来实现异步事务性流量(主要是金融交易,其中对可跟踪、可靠、ACID 类型消息传递的需求超过了对事务延迟的需求)。

我们发现 MSMQ 的优点是:

  • 事务传递 - 消息可以由目标系统拉取并在 2 阶段 UOW 下插入到 SQL 中。
  • Hugh 的关于传递与系统可用性脱钩的观点(如果目标系统在不合理的时间内停机,您仍然拥有死信队列)
  • 负载平衡/限制 - 目标系统可以通过将消息从队列中拉出来防止过度热心的消息传递以更均匀的速度。
  • 审核 - 使用 MSMQ 上的日志允许额外的跟踪层。
  • 另请注意,MSMQ 有一个 WCF 适配器 - 不需要自定义侦听器。

我们通常避免直接从 BizTalk 调用 SQL:

  • 读取此内容相当于轮询数据库,希望有消息准备发送(这可能会产生与调用频率相关的问题,即冗余、引起的延迟和负载) SQL 和争用 - 例如,在应用程序将数据添加到表时进行轮询 我们宁愿让每个应用程序决定何时向 BizTalk/ESB 提交消息
  • 以进行写入操作,除非将数据卸载到暂存区域以供处理。目的地应用程序中,它可能会导致大部分“业务”处理转移到 BizTalk 中(即验证、应用业务规则等) - 恕我直言,这对于 BizTalk 来说太细粒度了,而且正如您所发现的,它可能很难控制。消息传递到 SQL 的速率(例如,除非您开始使用 Singleton Orhcestrations 等),这又会导致锁定/争用问题。

I'm with Hugh - we've used MSMQ (and IBM MQ Series) successfully with BizTalk for asynchronous, transactional traffic (mostly financial transactions, where the need for traceable, reliable, ACID type message delivery outweighs any need for transaction latency).

We've found the benefits of MSMQ to be:

  • Transactional delivery - messages can be pulled off by the destination system and inserted into SQL under a 2 phase UOW.
  • Hugh's point about delivery decoupled from system availability (and you still have the Dead Letter Queue if the target system is down for an unreasonable period of time)
  • Load balancing / throttling - a destination system can protect against overzealous message delivery by pulling messages off the queue at a more even pace.
  • Auditing - using the journalling on MSMQ allows an additional layer of tracing.
  • Also note that there is a WCF adapter for MSMQ - no requirement for custom listeners.

We generally stay away from calling SQL directly from BizTalk:

  • For reading this equates to polling the database in the hope that there are messages ready to be sent (this can create issues relating to frequency of calling, i.e. redundancy, induced latency, and load on SQL, and contention - e.g. polling while data is being added by an app to the tables. We would rather have each app decide when to submit messages to BizTalk / ESB.
  • for write operations, unless data is offloaded into a staging area for processing by destination apps, it can lead to much of the 'business' processing moving into BizTalk (i.e. validation, applying business rules etc) - IMHO this is too fine-grained for BizTalk. And as you've found, it can be hard to control the rate of message delivery into SQL (e.g. unless you start using Singleton Orhcestrations etc), which again causes locking / contention issues.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文