使用 Kernel#fork 进行后台进程,优点吗?缺点?

发布于 2024-08-08 15:46:40 字数 416 浏览 9 评论 0原文

我想知道使用 fork{} 来“后台”rails 应用程序中的进程是否是一个好主意......

根据我收集的信息 fork{my_method; Process#setsid} 实际上做了它应该做的事情。

1) 创建具有不同 PID 的另一个进程

2) 不中断调用进程(例如,它继续运行,无需等待 fork 完成)

3) 执行子进程直到它完成

..这很酷,但它是一个好主意? fork 究竟在做什么?它是否会在内存中创建整个 Rails mongrel/passenger 实例的重复实例?如果是这样那就太糟糕了。或者,它是否以某种方式在不消耗大量内存的情况下做到这一点。

我的最终目标是废除我的后台守护进程/队列系统,转而分叉这些进程(主要是发送电子邮件)——但如果这不能节省内存,那么这绝对是朝着错误方向迈出的一步

I'd like some thoughts on whether using fork{} to 'background' a process from a rails app is such a good idea or not...

From what I gather fork{my_method; Process#setsid} does in fact do what it's supposed to do.

1) creates another processes with a different PID

2) doesn't interrupt the calling process (e.g. it continues w/o waiting for the fork to finish)

3) executes the child until it finishes

..which is cool, but is it a good idea? What exactly is fork doing? Does it create a duplicate instance of my entire rails mongrel/passenger instance in memory? If so that would be very bad. Or, does it somehow do it without consuming a huge swath of memory.

My ultimate goal was to do away with my background daemon/queue system in favor of forking these processes (primarily sending emails) -- but if this won't save memory then it's definitely a step in the wrong direction

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

神经暖 2024-08-15 15:46:40

分叉确实会复制您的整个过程,并且根据您与应用程序服务器的具体连接方式,也会复制该过程。正如其他讨论中所指出的,这是通过写时复制来完成的,因此这是可以容忍的。毕竟,Unix 是围绕 fork(2) 构建的,因此它必须相当快地管理它。请注意,任何部分缓冲的 I/O、打开的文件和许多其他内容也会被复制,以及弹簧加载以写出它们的程序的状态,这是不正确的。

我有一些想法:

  • 您使用 Action Mailer 吗?看起来电子邮件可以很容易地通过 AM 或通过 Process.popen 之类的东西来完成。 (Popen 会执行一个 fork,但紧接着执行一个 exec。)
  • 立即通过执行另一个 ruby​​ 解释器的 Process.exec 加上您的功能来摆脱所有状态。如果有太多状态需要传输,或者您确实需要使用这些重复的文件描述符,您可以执行类似 IO#popen 的操作,以便可以发送子进程要做的工作。系统会自动与父进程共享包含子进程的Ruby解释器文本的页面。
  • 除了上述之外,您可能还需要考虑使用 daemons gem。虽然您的 Rails 进程已经是一个守护进程,但使用 gem 可能会让您更轻松地保持一个后台任务作为批处理作业服务器运行,并且可以轻松启动、监视、在崩溃时重新启动以及在崩溃时关闭。 exit
  • 如果确实从 fork(2)ed 子进程中退出,请使用 exit! 而不是已设置消息队列和守护进程的
  • 起来,就像你一样,对我来说听起来像是一个很好的解决方案:-)

The fork does make a copy of your entire process, and, depending on exactly how you are hooked up to the application server, a copy of that as well. As noted in the other discussion this is done with copy-on-write so it's tolerable. Unix is built around fork(2), after all, so it has to manage it fairly fast. Note that any partially buffered I/O, open files, and lots of other stuff are also copied, as well as the state of the program that is spring-loaded to write them out, which would be incorrect.

I have a few thoughts:

  • Are you using Action Mailer? It seems like email would be easily done with AM or by Process.popen of something. (Popen will do a fork, but it is immediately followed by an exec.)
  • immediately get rid of all that state by executing Process.exec of another ruby interpreter plus your functionality. If there is too much state to transfer or you really need to use those duplicated file descriptors, you might do something like IO#popen instead so you can send the subprocess work to do. The system will share the pages containing the text of the Ruby interpreter of the subprocess with the parent automatically.
  • in addition to the above, you might want to consider the use of the daemons gem. While your rails process is already a daemon, using the gem might make it easier to keep one background task running as a batch job server, and make it easy to start, monitor, restart if it bombs, and shut down when you do...
  • if you do exit from a fork(2)ed subprocess, use exit! instead of exit
  • having a message queue and a daemon already set up, like you do, kinda sounds like a good solution to me :-)
梓梦 2024-08-15 15:46:40

请注意,它会阻止您使用 JRuby on Rails,因为 fork() 尚未实现。

Be aware that it will prevent you from using JRuby on Rails as fork() is not implemented (yet).

绅士风度i 2024-08-15 15:46:40

fork 的语义是将进程的整个内存空间复制到新进程中,但许多(大多数?)系统将通过仅复制虚拟内存表并将其标记为写入时复制来实现这一点。这意味着(至少一开始)它不会使用更多的物理内存,仅足以创建新表和其他每个进程的数据结构。

也就是说,我不确定 Ruby、RoR 等与写时复制分叉的交互效果如何。特别是,如果垃圾收集涉及许多内存页面(导致它们被复制),则可能会出现问题。

The semantics of fork is to copy the entire memory space of the process into a new process, but many (most?) systems will do that by just making a copy of the virtual memory tables and marking it copy-on-write. That means that (at first, at least) it doesn't use that much more physical memory, just enough to make the new tables and other per-process data structures.

That said, I'm not sure how well Ruby, RoR, etc. interacts with copy-on-write forking. In particular garbage collection could be problematic if it touches many memory pages (causing them to be copied).

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文