当 cron 作业不够时我该使用什么? (php)

发布于 2024-08-29 19:46:07 字数 249 浏览 13 评论 0原文

我正在尝试找出每天运行数千次相当繁重的 PHP 任务的最有效方法。它需要与 Gmail 建立 IMAP 连接、循环电子邮件、将此信息保存到数据库并在本地保存图像。

使用 cron 经常运行这个任务并不是什么大不了的事,但我需要每分钟运行一次,而且我知道最终 cron 会开始相互运行并导致内存问题。

当您需要每分钟多次高效运行某项任务时,下一步是什么?我一直在读有关豆茎和豆茎的文章。 pheanstalk,我不完全确定这是否能满足我的需要。想法???

I'm trying to figure out the most efficient way to running a pretty hefty PHP task thousands of times a day. It needs to make an IMAP connection to Gmail, loop over the emails, save this info to the database and save images locally.

Running this task every so often using a cron isn't that big of a deal, but I need to run it every minute and I know eventually the crons will start running on top of each other and cause memory issues.

What is the next step up when you need to efficiently run a task multiple times a minute? I've been reading about beanstalk & pheanstalk and I'm not entirely sure if that will do what I need. Thoughts???

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

倒带 2024-09-05 19:46:07

我不是 PHP 专家,但是...是什么阻止您将脚本作为守护进程运行?我已经编写了很多 Perl 脚本来实现这一点。

I'm not a PHP guy but ... what prevents you from running your script as a daemon? I've written many a perl script that does just that.

莳間冲淡了誓言ζ 2024-09-05 19:46:07

要么创建一个锁定机制,这样脚本就不会重叠。这非常简单,因为脚本仅每分钟运行一次,一个简单的 .lock 文件就足够了:

<?php
  if (file_exists("foo.lock")) exit(0);
  file_put_contents("foo.lock", getmypid());

  do_stuff_here();

  unlink("foo.lock");
?>

这将确保脚本不会并行运行,您只需确保在程序退出时删除 .lock 文件,这样您就可以应该有一个出口点(除了开头的出口)。

正如 Brian Roach 所建议的那样,一个好的替代方案是始终运行并保持与 IMAP 服务器的连接的专用服务器进程。这大大减少了开销,并且并不比编写普通的 php 脚本难多少:

<?php
  connect();
  while (is_world_not_invaded_by_aliens())
  {
    get_mails();
    get_images();
    sleep(time_to_next_check());
  }
  disconnect();
?>

Either create a locking mechanism so the scripts won't overlap. This is quite simple as scripts only run every minute, a simple .lock file would suffice:

<?php
  if (file_exists("foo.lock")) exit(0);
  file_put_contents("foo.lock", getmypid());

  do_stuff_here();

  unlink("foo.lock");
?>

This will make sure scripts don't run in parallel, you just have to make sure the .lock file is deleted when the program exits, so you should have a single point of exit (except for the exit at the beginning).

A good alternative - as Brian Roach suggested - is a dedicated server process that runs all the time and keeps the connection to the IMAP server up. This reduces overhead a lot and is not much harder than writing a normal php script:

<?php
  connect();
  while (is_world_not_invaded_by_aliens())
  {
    get_mails();
    get_images();
    sleep(time_to_next_check());
  }
  disconnect();
?>
_蜘蛛 2024-09-05 19:46:07

我有很多这样的脚本,我不想从 cron 运行它们,以防它们堆积起来。

#!/bin/sh
php -f fetchFromImap.php
sleep 60
exec $0

exec $0 部分再次启动脚本运行,并在内存中替换自身,因此它将永远运行而不会出现问题。 PHP 脚本使用的任何内存在退出时都会被清理,所以这也不是问题。

简单的一行就可以启动它,并将其置于后台:

cd /x/y/z ; nohup ./loopToFetchMail.sh &

或者可以在机器启动时通过各种方式类似地启动它(例如 Cron 的“@reboot ....”)

I've got a number of scripts like these, where I don't want to run them from cron in case they stack-up.

#!/bin/sh
php -f fetchFromImap.php
sleep 60
exec $0

The exec $0 part starts the script running again, replacing itself in memory, so it will run forever without issues. Any memory the PHP script uses is cleaned up whenever it exits, so that's not a problem either.

A simple line will start it, and put it into the background:

cd /x/y/z ; nohup ./loopToFetchMail.sh &

or it can be similarly started when the machine starts with various means (such as Cron's '@reboot ....')

挖鼻大婶 2024-09-05 19:46:07

fcron http://fcron.free.fr/ 如果旧作业仍在运行,则不会启动新作业,您可以使用 @ 1 命令 而不必担心竞争条件。

fcron http://fcron.free.fr/ will not start new job if old one is still running, Your could use @ 1 command and not worry about race conditions.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文