在 .NET 服务总线队列与 Azure 队列服务之间进行选择

发布于 2024-08-15 01:59:59 字数 1706 浏览 10 评论 0原文

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(6

故乡的云 2024-08-22 01:59:59

存储队列与服务总线

以下是我在思考这个问题时所考虑的一些不同考虑因素的细分。

可用性

自去年 11 月存储中断以来,Azure 承诺他们将永远不会再次将代码部署到所有区域 - 将其内置到系统中以使其不可能。 https://azure.microsoft.com/en-us/blog/final-root-cause-analysis-and-improvement-areas-nov-18-azure-storage-service-interruption/

msdn 关于可用性的说法如下:

如果您已经在使用 Azure 存储 Blob 或表并开始使用队列,则可以保证 99.9% 的可用性。如果将 Blob 或表与服务总线队列一起使用,可用性将会较低。

Azure 队列旨在支持应用程序组件的解耦,以提高可扩展性和故障容忍度。

就个人开发而言

,我对存储 API 很满意,并且大多数应用程序的其他区域都需要它的 blob 存储。存储队列使用与存储 blob 完全相同的 sdk。
Azure 队列提供了跨队列、表和 BLOB 的统一且一致的编程模型

。成本

服务总线支持的接收和删除模式提供了减少消息传递操作计数(和相关成本)的能力,以换取降低的交付保证。

似乎有一些围绕服务总线成本控制的工具,如果您必须开始保留运行应用程序的预算,您可以利用它们 - 我尝试分解下面的潜在存储队列成本:)。 GRS 每小时排队超过 40,000 人,每月费用不到 100 美元。与我们其余的存储成本放在一起,我认为专注于削减这里的成本并没有什么好处。 (两者的带宽相同,比较时会自行抵消)

存储定价

您可以获得无限的免费队列和操作 - 您为空间付费

  • 假设平均消息大小为 30K
  • 假设 1000K(MB 而非 1024)
  • 假设您不会达到分级定价above 1TB

30K / 1 message * 1 TB / 1000000000K * $.095 / 1 GB * 1000GB / 1 TB = $0.00000285 / message for the first TB of use

1 message / ~30K * 1000000000K / 1 TB = 33333333 messages in a TB

33333333消息 * $0.00000285 / 消息 = ~$95 美元,第一个 TB

分布在一个月内,我们可以使用第一个 TB

服务总线

  • 每小时处理 40,000 条消息,每月 10 美元,
  • 按操作支付基本价格(任何 api 调用都是操作) - 添加队列/接收队列/监控队列等。
  • 1250 万个免费操作,
  • 之后按每百万个操作

付费

获得

您每月将 通过在检索消息时指定消息计数,最多可接收 32 条消息,而服务总线允许队列客户端将多个消息批处理到单个发送操作中。

所以存储是批量接收,而服务总线是批量发送。

通过监视

Azure 队列,您可以获得针对队列执行的所有事务的详细日志以及聚合指标。这种支持并不是随服务总线而来的,但可能可以在某处找到预构建的解决方案。

转发

服务总线具有存储队列缺失的自动转发功能。

自动转发使数千个队列能够将其消息自动转发到单个队列,接收应用程序从中使用消息。您可以使用此机制来实现安全性、控制流以及隔离每个消息发布者之间的存储。

重复

服务总线队列支持的重复检测功能会根据以下条件自动删除发送到队列或主题的重复消息: MessageID 属性的值。

存储队列消息可能会重复,而不会发出警告。

元数据

服务总线为您提供消息头+正文的两部分。对于全球部署的基础设施来说,这是一个非常有用的功能。让您可以使用区域名称和实例 ID 等内容来装饰消息。队列消息是简单的字符串。另一方面,Azure 存储队列以名称/值对的形式提供对可应用于队列描述的任意属性的支持。因此,您可以在 Service Bus 中装饰消息,并用存储队列装饰队列。

交付保证

Service Bus 提供最多一次和至少一次交付,而队列仅提供至少一次交付。如果并发订阅者出现问题,这可能会限制我们使用队列的能力。

性能

Azure 存储队列提供 10 毫秒的延迟(在数据中心内),而服务总线提供 20-25 毫秒的延迟。服务总线确实提供长轮询,如果您需要的话,它甚至会比 10 毫秒更好。

安全

存储队列使用主/辅助共享密钥,而服务总线通过具有发送者/接收者/管理员角色的 Active Directory 提供 RBAC。

参考文献

Storage Queues vs Service Bus

Here is a breakdown of some of the different considerations I had in thinking through this question.

Availability

Since the storage outage last November Azure promised they would never deploy code to all the regions at once again - built it into the system to make it impossible. https://azure.microsoft.com/en-us/blog/final-root-cause-analysis-and-improvement-areas-nov-18-azure-storage-service-interruption/

Here's what msdn says about availability:

If you are already using Azure Storage Blobs or Tables and you start using queues, you are guaranteed 99.9% availability. If you use Blobs or Tables with Service Bus queues, you will have lower availability.

Azure Queues are designed to support the decoupling of application components to increase scalability and tolerance for failures.

Development

Personally, I am comfortable with the storage APIs and already have need for its blob storage in other areas of most apps. Storage queues use the very same sdk as storage blobs.
Azure Queues provide a uniform and consistent programming model across queues, tables, and BLOBs

Cost

The Receive and Delete mode supported by Service Bus provides the ability to reduce the messaging operation count (and associated cost) in exchange for lowered delivery assurance.

It seems like there is some tooling around cost control for service bus that you could leverage if you had to start keeping a budget to run your app - I did an attempt at breaking down the potential storage queue costs below :). It comes out to less than a hundred bucks a month at more than 40,000 queues an hour for GRS. Grouped in with the rest of our storage costs I don't see benefit to focusing on cutting costs here. (Bandwidth is the same for both and cancels itself out when comparing)

Storage pricing

You get unlimited free queues and operations - you pay for space

  • assume 30K message size as an average
  • assume 1000K in an MB not 1024
  • assume you will not hit the graduated pricing above 1TB

30K / 1 message * 1 TB / 1000000000K * $.095 / 1 GB * 1000GB / 1 TB = $0.00000285 / message for the first TB of use

1 message / ~30K * 1000000000K / 1 TB = 33333333 messages in a TB

33333333 messages * $0.00000285 / message = ~$95 dollars for the first TB

spread out over a month we can do like 40,000 messages an hour with that 1st TB

Service Bus pricing

  • 10 bucks a month base price
  • pay per operation (any api call is an op) - add a queue / receive a queue / monitor the queue / etc.
  • you get 12.5 million free ops / month
  • pay per million ops after that

Hard to estimate usage here but 100 million operations costs 80 bucks a month

Batch Receive

Storage can batch up to a maximum of 32 messages by specifying message count when retrieving messages while Service Bus allows a queue client to batch multiple messages into a single send operation.

So storage is batch receive while service bus is batch send.

Monitoring

Azure Queues enable you to obtain a detailed log of all of the transactions executed against the queue, as well as aggregated metrics. This kind of support does not come out of box with Service Bus - but could probably find a prebuilt solution somewhere.

Forwarding

Service Bus has an auto-forwarding feature that storage queues is missing.

auto-forwarding enables thousands of queues to auto-forward their messages to a single queue, from which the receiving application consumes the message. You can use this mechanism to achieve security, control flow, and isolate storage between each message publisher.

Duplicates

The duplication detection functionality supported by Service Bus queues automatically removes duplicate messages sent to a queue or topic, based on the value of the MessageID property.

Storage queue messages can duplicate without warning.

Metadata

Service Bus gives you 2 parts of a message header+body. This is a very useful feature for a globally deployed infrastructure. Letting you decorate your messages with things like region name and instance id. Queue messages are simple strings. On the other hand Azure storage queues provide support for arbitrary attributes that can be applied to the queue description, in the form of name/value pairs. So you can decorate the message in Service Bus and decorate the queue with storage queues

Delivery Guarantee

Service Bus offers At-Most-Once and At-Least-Once while Queues only offer At-Least-Once delivery. This could limit our ability to use queues if concurrent subscribers are ever a problem.

Performance

Azure storage queues offer a 10ms latency (within a datacenter) while Service Bus 20-25ms latency. Service bus does offer long-polling which would be even better than 10ms if you have a need for it.

Security

Storage queues use the primary/secondary shared key thing while service bus provides RBAC via Active Directory with Sender/Receiver/Admin roles.

references

娇纵 2024-08-22 01:59:59

我建议您坚持使用 Azure 队列在 Web 角色和辅助角色之间进行通信。使用队列是 Azure 进程之间通信的官方且认可的方式,我真诚地怀疑您是否会将自己编程到一个角落。服务总线 (AppFabric) 的开销较高,虽然非常适合与外部应用程序通信,但对于 Azure 应用程序中的快速而简单的消息来说可能不是最佳选择。

I would recommend that you stick with Azure Queues for communicating between web and worker roles. Using queues is the official and sanctioned way of communicating between Azure processes and I sincerely doubt that you will program yourself into a corner. Service Bus (AppFabric) has a higher overhead and although really good for talking to external apps, may not be optimal for quick and simple messages within your Azure app.

那些过往 2024-08-22 01:59:59

据我了解,服务总线(原样)已经有一段时间排队了,但这些并不能保证传递消息 - 好机会!

As I understand it, the Service Bus (as was) has had queues for a while, but these are not guaranteed to deliver the message - bon chance!

对岸观火 2024-08-22 01:59:59

Azure 队列接缝匹配,因为您的用例仍然是基本的,具有简单的基于 REST 的 Get/Put/Peek 界面。

该文档最近已更新(05/21/2015),它详细介绍了使用其中一种或另一种的情况,以及常见功能(事务支持、队列和消息的大小、生存时间等):

< a href="https://azure.microsoft.com/en-us/documentation/articles/service-bus-azure-and-service-bus-queues-compared-contrasted/" rel="nofollow">https:// /azure.microsoft.com/en-us/documentation/articles/service-bus-azure-and-service-bus-queues-compared-contrasted/

Azure Queues seams to match because your use case remains basic with simple REST-based Get/Put/Peek interface.

The documentation havs been updated recently (05/21/2015) and it details precisely when using the one or the other, and the common features (transaction support, size of the queue and message, Time to Live, ...) :

https://azure.microsoft.com/en-us/documentation/articles/service-bus-azure-and-service-bus-queues-compared-contrasted/

失去的东西太少 2024-08-22 01:59:59

开发人员学习的队列相关模式可以应用于两者。
从可靠性和易于实施的角度来看,两者都可以使用。

只有存储队列才能做的事情
1)处理消息的worker崩溃。后续工作人员想要读取消息的状态以从前一个工作人员停止的地方继续。
2) 您需要针对队列执行的所有事务的服务器端日志。

但比较并不重要。如果我们需要自定义队列开发,那么请始终使用存储队列。它是第一个由微软开发的。服务总线是复制BizTalk引入的,目的是集成(混合),因此这一行有高级功能:会话、事务、自动死信等。

此链接提供了比较,因此

The queue related patterns a developer learns can be applied to both.
Both can be used from a reliability and implementation-easiness point of view.

Things only Storage queue can do
1) The worker processing a message crashes. A subsequent worker wants to read the status of the message to continue from where the prior worker left off.
2) You require server side logs of all of the transactions executed against your queues.

But comparisons don't matter. If custom queue development is what we need then always use storage queue. It was the first to be developed by Microsoft. Service bus was brought-in copying BizTalk and the purpose is integration(hybrid), hence there are advanced features in this line: sessions, transactions, automatic dead-letters, etc.

This link provides a comparison , so does this link. It will be difficult to analyze everything and start in an agile development hence the above mentioned rule.

撩起发的微风 2024-08-22 01:59:59

为了让事情变得非常清楚,这是两个 Azure 组件之间的比较,这两个组件是在不同的时间点、出于不同的原因创建的。

存储队列和服务总线队列 - 比较和对比

Azure 支持两种类型的队列机制:存储队列和服务总线
队列。

存储队列是 Azure 存储基础结构的一部分,
具有简单的基于 REST 的 GET/PUT/PEEK 接口,提供
服务内部和服务之间可靠、持久的消息传递。

服务总线队列是更广泛的 Azure 消息传递的一部分
支持排队以及发布/订阅的基础设施,以及
更先进的集成模式。有关服务的更多信息
总线队列/主题/订阅,请参阅服务总线概述。

虽然两种队列技术同时存在,但存储队列
首先被引入,作为建立在
Azure 存储服务的顶部。服务总线队列构建在
旨在集成更广泛的消息传递基础设施
应用程序或应用程序组件可能跨越多个
通信协议、数据合同、信任域和/或网络
环境。

To make things very clear, this is a comparison between two Azure components, created at a different point in time, for different reasons.

Storage queues and Service Bus queues - compared and contrasted

Azure supports two types of queue mechanisms: Storage queues and Service Bus
queues.

Storage queues, which are part of the Azure storage infrastructure,
feature a simple REST-based GET/PUT/PEEK interface, providing
reliable, persistent messaging within and between services.

Service Bus queues are part of a broader Azure messaging
infrastructure that supports queuing as well as publish/subscribe, and
more advanced integration patterns. For more information about Service
Bus queues/topics/subscriptions, see the overview of Service Bus.

While both queuing technologies exist concurrently, Storage queues
were introduced first, as a dedicated queue storage mechanism built on
top of Azure Storage services. Service Bus queues are built on top of
the broader messaging infrastructure designed to integrate
applications or application components that may span multiple
communication protocols, data contracts, trust domains, and/or network
environments.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文