处理数百万条定时（预定）消息的解决方案？
我正在评估处理大量排队消息的可能解决方案，这些消息必须在特定的日期和时间传递给工作人员。执行它们的结果主要是对存储数据的更新，它们最初可能会也可能不会由用户操作触发。
例如，考虑一下您在假设的大型星际争霸游戏服务器中实现的用于存储和执行用户操作的内容，例如升级建筑物、孵化士兵，所有这些都需要在几秒钟后应用于游戏状态或玩家启动后几分钟。
问题是我似乎找不到合适的术语来命名这个问题区域。有几个看起来相似但不同：
cron/task/job Scheduler
- 队列的内容不是动态的，而是预定义的。
- 每项任务均已安排。
消息队列
- 队列的内容是动态的。
- 每项任务都旨在立即交付。
???
- 队列的内容是动态的。
- 每项任务均已安排。
如果存在允许有条件传送消息的消息队列，那么可能就是这样。
摘要：
- 这种技术叫什么？
- 有哪些解决方案？
I'm evaluating possible solutions for handling a large quantity of queued messages, which must be delivered to workers at a certain date and time. The result of executing them is mostly updates to stored data, and they may or may not be originally triggered by user action.
For example, think of what you'd implement in a hypothetical large-scale StarCraft game server for storing and executing users' actions, like upgrading a building, hatching a soldier, all of which requires to be applied to the game state after several seconds or minutes after the player initiates them.
The problem is I can't seem to find the right term to name this problem area. There are several that looks similar, but different:
cron/task/job scheduler
- The content of the queue is not dynamic, it's predefined.
- Each task is scheduled.
message queue
- The content of the queue is dynamic.
- Each task is intended to be delivered immediately.
???
- The content of the queue is dynamic.
- Each task is scheduled.
If there are message queues that allow conditional delivery of messages, that might be it.
Summary:
- What are these kind of technology called?
- What are some of the solutions out there?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
从表面上看，这听起来像是一个微不足道的优先级队列。在这种情况下，优先级是完成时间，您可以检查队列的前面以查看下一个事件何时到期。几乎每种语言都带有优先级队列或可以轻松用作优先级队列的东西，所以我不确定这里的实际问题是什么。
当涉及到数百万条消息时，您是否担心可扩展性？显然，“数百万”是一个毫无意义的术语——如果每天数百万，那就是一个微不足道的问题。如果每秒数百万，那么您可以水平扩展，将队列拆分到多个进程。 （这种队列系统的好处是这种并行化非常简单。）
我敢打赌，在实现大型实时策略游戏服务器时，您早在开始遇到消息队列问题之前就会遇到网络问题。
This just sounds like a trivial priority queue on the surface. The priority in this case is the time of completion, and you check the front of the queue to see when the next event is due. Pretty much every language comes with a priority queue or something that can easily be used as one, so I'm not sure what the actual problem is here.
Is it that you're worried about scalability, when it comes to millions of messages? Obviously 'millions' is a meaningless term - if that's millions per day, it's a trivial problem. If it's millions per second, then you can just scale horizontally, splitting the queue across multiple processes. (And the benefit of such a queue system is that this parallelization is really simple.)
I would bet that when implementing a large scale real-time strategy game server you would hit networking problems long before you start hitting problems with the message queue.
您是否尝试过通过 Iron.io 查看推送队列？队列的内容可以是您喜欢的任何内容，并且您可以指定将消息推送到的 Webhook。您还可以为每条消息设置延迟。
尽管每个队列的 webhook 都是静态的，并且延迟并不总是完全准时（可能最多延迟一分钟）。如果时机更重要或者为每条消息提供不同的 Webhook 的能力很重要，请尝试查看 boomerang.io。
他们说他们在时间上非常准确，您可以为 webhook 返回提供延迟或 unix 时间戳，这是针对每条消息的。听起来其中任何一个都可能适合你。
Have you tried looking at push queues by Iron.io? The content of the queue can be anything you like, and you specify a webhook to where the messages will be pushed to. You can also set a delay for each of the messages.
The webhook is static though for each queue and delay isn't always exactly on time (could be up to a minute off). If timing is more important or the ability of providing a different webhook per message is important, try looking at boomerang.io.
They say they are pretty accurate on the timing, you can provide a delay or unix timestamp for the webhook to return and that is per message. Sounds like either of those might work for you.
对于星际争霸，我会使用红矮星服务器。
对于 Java EE 应用程序，我将使用 Quartz Scheduler。
For StarCraft, I would use the Red Dwarf server.
For a Java EE app, I would use Quartz Scheduler.
在我看来，基于队列的解决方案在这种情况下是最好的，原因有很多：
剩下的问题是队列中消息的立即传递。有两种方法可以解决此问题：延迟消息入队或延迟执行出队消息。我会采用第一种方法，延迟排队。
消息有两个属性：（内容、延迟）。您将消息提供给系统中的组件，该组件会在适当的时间对消息进行排队。
我不确定您使用的是什么编程语言，但 MS .NET 4 框架支持 这样的场景（任务延迟执行）。
It seems to me that a queue-based solution would be best in this case for a number of reasons:
Remaining problem is the immediate delivery of messages in the queue. You have two ways to solve this: either delay enqueuing of messages or delay execution of dequeued messages. I would go with the first approach, delayed enqueuing.
A message then has two properties: (content, delay). You provide the message to a component in your system that queues the message at the appropriate time.
I'm not sure what programming language you're using, but the MS .NET 4 framework has support for such a scenario (delayed execution of tasks).