需要很长执行时间的 php 脚本的 Cron 作业

发布于 2024-09-30 01:02:08 字数 298 浏览 4 评论 0 原文

我有一个作为 cron 作业运行的 php 脚本,它执行一组简单的任务,这些任务为数据库中的每个用户循环,大约需要 30 分钟才能完成。这个过程每小时都会启动一次,并且需要尽可能快速和高效。我遇到的问题就像任何服务器脚本一样,执行时间各不相同,我需要找出最佳的 cron 时间设置。

如果我每分钟运行 cron,我需要在该分钟结束前 20 秒停止脚本的最后一个循环,以确保当前循环及时完成。在一个小时的过程中,这会浪费很多时间。

我想知道简单地删除 php 执行时间限制并每小时运行一次脚本并让它运行完成是否是一个坏主意......这是一个坏主意吗?

I have a php script run as a cron job that executes a set of simple tasks that loops for each user in the database and takes about 30 mins to complete. This process starts over every hour and needs to be as fast and efficient as possible. The problem Im having, is like with any server script, execution time varies and I need to figure out the best cron time settings.

If I run cron every minute, I need to stop the last loop of the script 20 seconds before the end of the minute to make sure that the current loop finishes in time. Over the course of the hour this adds up to a lot of wasted time.

Im wondering if its a bad idea to simple remove the php execution time limit and run the script once an hour and let it run to completion.... is this a bad idea?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(8

妥活 2024-10-07 01:02:08

您还可以使用 set_time_limit() 重置计数器,而不是设置 max_execution_time在每个循环上。这将确保您的脚本永远不会耗尽时间,除非当前循环中有严重的问题(并且花费的时间超过 max_execution_time)。

基本上,这应该使您的脚本运行所需的时间,同时在两次 set_time_limit() 调用之间设置 30 秒的超时时间。

Instead of setting the max_execution_time you could also use set_time_limit() to reset the counter on every loop. This will ensure your script is never running out of time unless there is something serious hanging within the current loop (and taking longer than the max_execution_time).

Basically this should make your script run as long as it needs while giving it a 30 seconds timeout between two set_time_limit() calls.

别忘他 2024-10-07 01:02:08

假设您希望尽快完成工作,请不要使用 cron。 Cron 对于需要在特定时间发生的事情很有用。它经常被滥用来模拟后台进程,理想情况下,工作一出现就立即处理工作。您可能应该编写一个连续运行的守护进程。 (注意:您还可以查看消息/工作队列类型系统,也有很好的库可以执行此操作)

您可以使用 pcntl 函数(因为您不关心多个工作进程,所以它是 超级简单让进程在后台运行。),或者作弊并制作一个永远运行的脚本通过 screen 运行它,或者利用一些可靠的库代码,如 PEAR 的 系统:守护进程nanoserv

一旦守护进程处理完毕,您真正关心的就是拥有一个永远运行的循环。您需要注意脚本不会泄漏内存或消耗太多资源。

一般来说,你可以这样做:

<?PHP
// some setup code 
while(true){
    $todo = figureOutIfIHaveWorkToDo();
    foreach($todo as $something){
        //do stuff with $something
        //remember to clean up resources so you don't leak memory!
        usleep(/*some integer*/);
    }
    usleep(/* some other integer */); 
}

而且它会工作得很好。

Assuming you'd like the work done ASAP, don't use cron. Cron is good for things that need to happen at specific times. It's often abused to simulate a background process that would ideally process work as soon as work appears. You should probably write a daemon that runs continuously. (Note: you could also look at a message/work-queue type system, there are nice libraries out there to do this too)

You can write a daemon from scratch using the pcntl functions (since you don't care about multiple worker processes, it's super-easy to get a process running in the background.), or cheat and just make a script that runs forever and run it via screen, or leverage some solid library code like PEAR's System:Daemon or nanoserv

Once the daemonization stuff is taken care of, all you really care about is having a loop that runs forever. You'll want to take care that your script doesn't leak memory, or consume too many resources.

Generally, you can do something like:

<?PHP
// some setup code 
while(true){
    $todo = figureOutIfIHaveWorkToDo();
    foreach($todo as $something){
        //do stuff with $something
        //remember to clean up resources so you don't leak memory!
        usleep(/*some integer*/);
    }
    usleep(/* some other integer */); 
}

And it'll work pretty well.

脱离于你 2024-10-07 01:02:08

将时间限制设置为 0 并让它完成它的工作是基于 PHP 的 cronjobs 的典型做法(根据我的经验),但这也是您应该问自己一些重要问题的时候,例如“我应该在编译语言?”以及“我是否以最大效率使用所有工具(数据库等)?”

也就是说,也许比完全取消时间限制更好的是将其设置为您实际想要的上限。如果这意味着 48 分钟,则 set_time_limit(48 * 60);

Setting the time limit to 0 and letting it do its thing is fairly typical of PHP based cronjobs (in my experience), but this is also the point when you should ask yourself a few important questions, such as "Should I rewrite this job in a compiled language?" and "Am I using all of my tools (database, etc) to their maximum efficiency?"

That said, maybe better than completely removing the time limit would be to set it to the upper limit you actually want. If that means 48 minutes, then set_time_limit(48 * 60);

梦魇绽荼蘼 2024-10-07 01:02:08

我真的认为你不应该将超时设置为0,那只是自找麻烦。最多设置为 59*60 秒,但设置为 0 可能会导致安全问题,如果脚本挂起,它几乎会永远挂起,直到服务器主机停止执行。这样做被认为是不好的做法。

I really think you shouldn't set the time out to 0, that is just looking for trouble. At most, set it to 59*60 seconds, but setting it to 0 might cause security problems, if a script hangs, it will hang almost forever until the server host stops the execution. It is considered bad practice to do so.

带上头具痛哭 2024-10-07 01:02:08

我过去曾使用 php 命令行界面来执行类似的长时间运行的任务。您可能不想删除任何请求的执行时间限制。

I have used the php command-line interface for similar long running tasks in the past. You probably do not want to remove the execution time limit for any request.

稳稳的幸福 2024-10-07 01:02:08

如果花费时间超过一个小时的可能性很小的话,这听起来是个好主意。但请注意,错误的 bug 可能是使其花费比预期更长的时间的好方法。

为了避免各种令人讨厌的问题,您应该有一个包含脚本进程 ID 的保护文件。启动时,您应该检查以确保该文件不存在,或者如果确实存在,则文件中的进程 ID 不存在(通过kill(pid, 0) 调用)。如果满足这些条件,请使用脚本的 PID 创建一个新文件,并在完成后删除该文件。

这与许多守护进程用来确保它尚未运行的技巧相同。如果守护进程突然被杀死,该文件仍然存在,但其中进程的PID不太可能在运行。

Sounds like a great idea if there's little chance that it will take more than an hour. Note, however, that the wrong bug can be a really good way of making it take longer than expected..

To avoid all sorts of nasty problems, you should have a guard file with the process ID of the script. On startup, you should check to make sure the file doesn't exist, or if it does that the process ID in the file doesn't exist (through a kill( pid, 0 ) call). If these conditions are met, create a new file with the script's PID and delete the file when you're done.

This is the same trick that many daemons use to ensure it isn't already running. If the daemon was killed suddenly, the file will still exist but the PID of the process therein is unlikely to be running.

桃扇骨 2024-10-07 01:02:08

根据脚本的作用,如果删除时间限制,可能会导致问题。例如,如果您轮询一个在作业运行时没有响应的外部服务器,并且您的 cron 需要 2 小时而不是 30 分钟才能完成,您可能会启动一堆 PHP 进程,即使之前的进程还没有启动还没有完成。这可能会导致系统不稳定和崩溃。

您可能有两个选择:

  • 确保事先没有运行脚本的其他实例,否则在启动时退出()。
  • 考虑将您的 cronjob 更改为守护进程。

Depending on what your script does, it can lead to problems if you remove the time limit. If per example, you are polling an external server that is unresponsive while the job is running, and that your cron takes 2 hours instead of 30 minutes to complete, you may get a stack of PHP processes being fired up even if the previous ones haven't completed yet. This can cause system instability and crashes.

You probably have two options:

  • Make sure that no other instance of your script is running beforehand, otherwise exit() on start.
  • Consider changing your cronjob into a daemon.
享受孤独 2024-10-07 01:02:08

它必须像发条一样每小时运行一次吗?

如果不拆分工作(您提到这不仅仅是一项简单任务),那么每小时执行每项任务?

或者按用户分开,在上午进行,然后在下一个小时进行新西兰?

Does it have to run hourly like clockwork?

If not split the job (you mentioned it was more than one simple task) do each task every hour?

Or split it per user, do A-M on hour, then N-Z the next?

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文