ActiveRecord 在分叉/线程应用程序中运行时无法重新连接?

发布于 2024-10-28 06:06:30 字数 1669 浏览 0 评论 0 原文

这是一个开放式问题,所以我先概述一下问题。我们有一个 Resque 工作人员,他应该从队列中弹出数据同步作业,原因有两个,使用 Cron (并一遍又一遍地支付 Rails 环境启动时间)很糟糕,而替代队列,Github 在他们 宣布 Resque。此外,Redis 时间序列功能已经在我们的基础设施中发挥了重要作用,我们还将 TS 数据混入 RRDTool 和 RRDTool 中。等等...

问题来了,典型的工作间隔为三个小时(但工作人员可以随时安排工作......因此出现队列),PostgreSQL 服务器消失了。很容易治愈,我希望在相应的环境下设置 reconnect: true 将确保其按预期工作。.我在 很少 一些地方reconnect: true对于使用fork()的应用程序不起作用。当然,Resque 会启动它的工作人员......我不明白的部分是为什么 ActiveRecord 的重新连接无法在这些情况下工作情况?

我注意到 reconnect! 的实现 MySQL 适配器PostgreSQL 适配器 在 ActiveRecord 中是不同的......但无论哪种方式,我都希望 ActiveRecord reconnect: true 配置能够工作。

问题似乎很清楚,当子进程存在时,它会关闭父进程创建的文件句柄(从而挂起与数据库的连接) - 是否可以以 ActiveRecord 的方式关闭文件句柄无法识别连接已终止?

就其价值而言,我还可以在 Github 上找到一个 ActiveRecord 感知 fork(),作为粘贴 - 它是未经测试,但我认为它可以工作(还没有在当前的 Rails 上尝试过......)

我的问题更多,如果你fork(),为什么 AR 中的自动重新连接无法工作? (而且,最近 - 我不可能是唯一遇到这个问题的人;我把它归结为将 PGSQL 与 Resque 一起使用!)

Bit of an open-ended question here, so I'll outline the problem first. We have a Resque worker who is supposed to pop data-synchronisation jobs off a queue, the reasons are two-fold, using Cron (and paying Rails environment boot time over-and-over) sucks, and the alternative queues, well Github made a pretty good case against them when they announced Resque. Also, the Redis time-series functionality already plays a big part of our infrastructure, where we also shuffle TS data into RRDTool & etc...

Here's the problem, with a typical three-hours between jobs (but staff can schedule jobs at any time… hence the queue), the PostgreSQL server goes away. Easy enough to cure, I expected that setting reconnect: true under the corresponding environment would ensure that this worked as expected.. I read in a few places that reconnect: true won't work for applications that use fork(). Which, naturally Resque does, to start it's workers… the part I don't understand is why ActiveRecord's reconnect can't work under these circumstances?

I noticed the implementations of reconnect! the MySQL Adapter and the PostgreSQL Adapter in ActiveRecord are different… but either way I would expect the ActiveRecord reconnect: true configuration to work.

The problem seems to be clear enough, when the child process exists, it closes the file handles created by the parent (thus hanging up the connection to the database) - is it possible to close a filehandle in such a way that ActiveRecord doesn't recognise that the connection has been terminated?

There's also, for what it's worth, an ActiveRecord aware fork() which I was able to find on Github, as a pastie - it's untested, but I assume it works (haven't tried it with current Rails..)

My question is more, Why can't the automatic reconnect in AR work if you fork()? (and, latterly - I can't be the only person who is having this problem; I'm chalking it up to using PGSQL with Resque!)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

心头的小情儿 2024-11-04 06:06:30

虽然不完全是“为什么不能重新连接”的答案,但我认为可以帮助您的是在初始化阶段的某个位置放置以下代码:


Resque.after_fork do |job|
  ActiveRecord::Base.connection.reconnect!
end

更新:
关于重新连接 - 似乎它是 MySQL 唯一的功能。
这是它在 mysql 适配器中的使用方式:
https://github.com/rails /rails/blob/master/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb#L848
如您所见,它使用底层驱动程序的重新连接功能。
https://github.com/kwatch/mysql-ruby /blob/master/ext/mysql.c#L923

另一方面,postgresql 适配器不会对 reconnect 选项执行任何操作,您可以看到 https://github.com/rails/rails/blob/master /activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb

此外,PostgreSQL 站点明确表示“使用开放的 libpq 连接分叉进程可能会导致不可预测的结果” - http://www.postgresql.org/docs/9.0/interactive/libpq-connect.html
而且很明显,C 驱动程序不提供任何重新连接功能。

While not exactly the answer to "why can't it reconnect", but I think that what can help you is the following code put somewhere in the initialization phase:


Resque.after_fork do |job|
  ActiveRecord::Base.connection.reconnect!
end

Update:
Regarding the reconnect - it seems that it's MySQL only feature.
This is how it's used in mysql adapter:
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb#L848
As you can see, it uses the underlying driver's reconnect feature.
https://github.com/kwatch/mysql-ruby/blob/master/ext/mysql.c#L923

On the other hand, the postgresql adapter doesn't do anything with regard to reconnect option, you can see the https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb

Also, the PostgreSQL site explicitly says that "Forking a process with open libpq connection can lead to unpredictable results" - http://www.postgresql.org/docs/9.0/interactive/libpq-connect.html
And it's also evident that the C driver doesn't provide any reconnect feature.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文