ruby 中进程间通信有哪些可用的消息解决方案?

发布于 2024-11-14 10:39:27 字数 455 浏览 6 评论 0原文

我有一个使用delayed_job的rails应用程序。我需要我的工作相互沟通,以处理诸如“任务 5 已完成”或“这是任务 5 需要处理的事项列表”之类的事情。

现在我有一个专门用于此目的的特殊表,并且我总是在事务内访问该表。运行良好。我想为其构建一个更干净的 api/dsl,但首先想检查是否已经有现有的解决方案。奇怪的是,我没有找到任何东西,我要么谷歌搜索完全错误,要么任务太简单(在事务中设置和获取值)以至于没有人将其抽象出来。

我错过了什么吗?

澄清:我不是在寻找新的排队系统,而是在寻找一种后台任务相互通信的方法。基本上只是安全地共享变量。以下框架是否提供此功能?遗憾的是,延迟的工作没有。

用例:“并行执行这 5 项任务,然后当它们全部完成后,执行最后一项任务。”因此,5 个任务中的每一个都会检查它是否是最后一个任务,如果是,则触发最后一个任务。

I have a rails app using delayed_job. I need my jobs to communicate with each other for things like "task 5 is done" or "this is the list of things that need to be processed for task 5".

Right now I have a special table just for this, and I always access the table inside a transaction. It's working fine. I want to build out a cleaner api/dsl for it, but first wanted to check if there were existing solutions for this already. Weirdly I haven't found a single things, I'm either googling completely wrong, or the task is so simple (set and get values inside a transaction) that no one has abstracted it out yet.

Am I missing something?

clarification: I'm not looking for a new queueing system, I'm looking for a way for background tasks to communicate with one another. Basically just safely shared variables. Do the below frameworks offer this facility? It's a shame that delayed job does not.

use case: "do these 5 tasks in parallel, and then when they are all done, do this 1 final task." So, each of the 5 tasks checks to see if it's the last one, and if it is, it fires off the final task.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

Bonjour°[大白 2024-11-21 10:39:27

我使用 resque。另外还有很多插件,这应该使进程间通信变得更容易。

使用 redis 还有另一个优点:您可以使用发布-订阅通道在工作人员/服务之间进行通信。

另一种方法(但未经我测试):http://www.zeromq.org/,它也有 < a href="http://www.zeromq.org/bindings%3aruby" rel="nofollow">Ruby 绑定。如果您喜欢测试新东西,请尝试 Zeromq。


更新

澄清/解释/扩展我上面的评论:

为什么我应该从DelayedJobs切换到Resque是提到的优势,我有队列和消息在一个系统中,因为 Redis 提供了这一点。

更多来源:

如果我必须留在 DJ 我会延长使用 rediszeromq/0mq(此处仅为示例)的工作线程类在我现有的后台作业中获取消息传递。

不会尝试使用ActiveRecord/MySQL进行消息传递(实际上甚至没有排队!),因为此数据库不是用于此用途的最佳性能系统尤其是当应用程序有太多后台工作人员、巨大的队列以及短时间内无数的消息交换时。

如果它是一个具有较少工作人员的小型应用程序,您也可以通过 DB 实现简单的消息传递,但在这里我更喜欢 memcache;消息是短暂的数据块,只能在内存中处理。

共享变量永远不会是一个好的解决方案。想象一下您的应用程序和工作人员可以运行的多台机器。如何确保它们之间保存变量传输?

好吧,有人可能会提到DRb(分布式红宝石),但它似乎不再被真正使用。 (到目前为止还没有见过现实世界的例子)

如果你想尝试一下 DRb,请阅读

我个人的偏好顺序:消息(真实)>数据库驱动的消息传递>变量共享

I use resque. Also there are lots of plugins, which should make inter-process comms easier.

Using redis has another advantage: you can use the pub-sub channels for communication between workers/services.

Another approach (but untested by me): http://www.zeromq.org/, which also has ruby bindings. If you like to test new stuff, then try zeromq.


Update

To clarify/explain/extend my comments above:

Why I should switch from DelayedJobs to Resque is the mentioned advantage that I have queue and messages in one system because Redis offers this.

Further sources:

If I had to stay on DJ I would extend the worker classes with redis or zeromq/0mq (only examples here) to get the messaging in my extisting background jobs.

I would not try messaging with ActiveRecord/MySQL (not even queueing actually!) because this DB isn't the best performing system for this use case especially if the application has too many background workers and huge queues and uncountable message exchanges in short times.

If it is a small app with less workers you also could implement a simple messaging via DB, but also here I would prefer memcache instead; messages are short living data chunk which can be handled in-memory only.

Shared variables will never be a good solution. Think of multiple machines where your application and your workers can live on. How you would ensure a save variable transfer between them?

Okay, someone could mention DRb (distributed ruby) but it seems not really used anymore. (never seen a real world example so far)

If you want to play around with DRb however, read this short introduction.

My personal preference order: Messaging (real) > Database driven messaging > Variable sharing

蒲公英的约定 2024-11-21 10:39:27

您可以使用管道:

reader, writer = IO.pipe

fork do
  loop do
    payload = { name: 'Kris' }
    writer.puts Marshal.dump(payload)
    sleep(0.5)
  end
end

loop do
  begin
    Timeout::timeout(1) do
      puts Marshal.load(reader.gets) # => { name: 'Kris' }
    end
  rescue Timeout::Error
    # no-op, no messages to receive
  end
end
  • 一种方式
  • 作为字节流读取

管道表示为一对,即读取器和写入器。要获得双向通信,您需要两组管道。

You can use Pipes:

reader, writer = IO.pipe

fork do
  loop do
    payload = { name: 'Kris' }
    writer.puts Marshal.dump(payload)
    sleep(0.5)
  end
end

loop do
  begin
    Timeout::timeout(1) do
      puts Marshal.load(reader.gets) # => { name: 'Kris' }
    end
  rescue Timeout::Error
    # no-op, no messages to receive
  end
end
  • One way
  • Read as a byte stream

Pipes are expressed as a pair, a reader and a writer. To get two way communication you need two sets of pipes.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文