如何在 Redis 中实现条件弹出并发友好?

发布于 2024-12-01 01:38:27 字数 319 浏览 0 评论 0原文

我正在构建类似延迟线的东西:一个进程将对象推送到列表中,另一个进程以相同的顺序将它们取出。

诀窍是对象只能在添加后一小时从列表中弹出。我可以为此使用时间戳。所有项目的延迟都是相同的并且永远不会改变。

现在,如何以并发友好的方式实现弹出(以便当多个工作人员访问该列表时它仍然有效)?我可以取出一个项目,检查时间戳,如果还为时过早,则将其放回列表中。但如果多个工人同时进行,可能会打乱物品的顺序。我可以检查第一个项目,只有在到期时才将其弹出。但当时可能有另一个工人把它弹出了,所以我弹出了错误的。

我应该使用 WATCH 命令吗?如何?我应该使用排序集而不是列表吗?帮助表示赞赏!

I am building something like a delay-line: one process RPUSHes objects into a list, another LPOPs them out in the same order.

The trick is that objects should only be popped from the list one hour after they have been added. I could use a time-stamp for that. The delay is the same for all items and never changes.

Now how to I implement the pop in a concurrency-friendly way (so that it still works when several workers access that list)? I could take out an item, check the timestamp and put it back into the list if it's still too early. But if several workers do that simultaneously, it may mess up the order of items. I could check the first item and only pop it if it's due. But another worker might have popped it then so I pop the wrong one.

Should I use the WATCH command? How? Should I use sorted sets instead of a list? Help appreciated!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

草莓酥 2024-12-08 01:38:27

我建议使用排序集。条目进入 zset,以普通标识符作为键,以 Unix 风格的时间戳作为分数。将时间戳设置为日期+时间,之后每个条目都准备好进行解析。工作人员执行 ZPOP,它不是内置的,但可以通过以下方式进行模拟:

MULTI
ZRANGE <key> 0 0 WITHSCORES
ZREMRANGEBYRANK <key> 0 0
EXEC

捕获 ZRANGE 的结果,并且您可以在时间,已经从集合中删除,及其分数。如果它还无效,请使用 ZADD将其放回原处。 <分数> <项目>。

I'd suggest using a sorted set. Entries go into the zset with the normal identifier as key and a Unix-style timestamp as the score. Make the timestamps the date+time after which each entry is ready for parsing. Workers do a ZPOP, which isn't a built-in but can be emulated with:

MULTI
ZRANGE <key> 0 0 WITHSCORES
ZREMRANGEBYRANK <key> 0 0
EXEC

Capture the results of the ZRANGE and you have the element with the lowest score at the time, already removed from the set, with its score. Put it back if it's not valid yet with a ZADD <key> <score> <item>.

日久见人心 2024-12-08 01:38:27

检查后将项目放回列表中不会使顺序混乱到足以产生影响的地步——通过任何类型的并发,您接受顺序不会被严格定义。但是,这种方法不会特别有效,因为如果不对项目进行一些处理,您就无法检查时间戳。

通过脚本构建,您可以为排序集实现条件弹出命令,尽管我认为这不是最简单的解决方案。

有几种方法可以使用多个列表来实现基本调度:

  • 将任务添加到排序集中,并让一个工作人员负责将它们从排序集中移动到多个工作人员通过 pop 访问的列表。
  • 每分钟列出一个清单,并让工作人员阅读过去的清单。您可以使用另一个跟踪最旧的非空队列的键来最大程度地减少检查的列表数量。

Putting the item back in the list after checking it won't mess up the ordering enough to matter - by having any sort of concurrency you accept that the order won't be strictly defined. However, that approach won't be particularly efficient as you can't check the timestamp without some processing of the item.

With the scripting build you could implement a conditional pop command for sorted sets, though I don't think that is the easiest solution.

There are a couple of ways to implement basic scheduling using multiple lists:

  • Add tasks to a sorted set and have a single worker responsible for moving them from the sorted set to a list that multiple workers access with pop.
  • Have a list for each minute, and have the workers read from past lists. You can minimize the number of lists checked by using another key that tracks the oldest non-empty queue.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文