在 php 中捕获 beanstalkd DEADLINE_SOON 事件
我在项目中使用 Beanstalkd 消息队列来处理 ffmpeg 视频。我使用 pheanstalk PHP 客户端作为 beanstalkd 队列。我的工作人员看起来像：
...
// get latest job
$job = $this->pheanstalk->reserve();
// get the data from the job
$jobData = unserialize($job->getData());
// process job, giving a result
$result = $this->task($jobData);
if ($result) {
// success = delete
$this->pheanstalk->delete($job);
} else {
// failed = bury
$this->pheanstalk->bury($job, 1024);
}
...
在 task() 方法中，我使用类似的方法：
// video processing
$processVideo = 'ffmpeg -vpre libx264-max -y -i inputfile ...';
shell_exec($processVideo);
// taking screenshots from video
...
如您所知，视频处理可能需要很长时间，有时它会超过预定义的作业ttr（运行时间）。这样，作业就会收到超时事件并再次返回队列。我同时运行我的工作程序多次以进行异步处理。因此，下一个自由工人试图承担未完成的工作。最后，我在一个视频文件下有两个或多个进程。 我认为，我可以定义很长的ttr，但这不是一个好的设计。我注意到，beanstalkd 有 DEADLINE_SOON 事件。但我不知道如何在我的工人中捕获它。我需要它能够使用 "touch" 命令。
您有什么建议吗？
I'm using Beanstalkd message queue for ffmpeg processing video in my project. I use pheanstalk PHP client for beanstalkd queue. My worker looks like:
...
// get latest job
$job = $this->pheanstalk->reserve();
// get the data from the job
$jobData = unserialize($job->getData());
// process job, giving a result
$result = $this->task($jobData);
if ($result) {
// success = delete
$this->pheanstalk->delete($job);
} else {
// failed = bury
$this->pheanstalk->bury($job, 1024);
}
...
In task() method I use smth like:
// video processing
$processVideo = 'ffmpeg -vpre libx264-max -y -i inputfile ...';
shell_exec($processVideo);
// taking screenshots from video
...
As you know, video processing can take long time, and sometimes it exceeds predefined job ttr (time to run). This way the job recieves time out event and comebacks to queue again. I run my worker several times at the same time for asynchronous processing. So the next free worker tries to take unfinished job. Finally, I have two or more processes under one video file.
I think, I can define very long ttr, but it is not a good desigion. I've noticed, beanstalkd has DEADLINE_SOON event. But I don't know, how to catch it in my worker. I need it to have an ability to use "touch" command.
Do you have any advices?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
根据邮件列表消息（此帖子），“
DEADLINE_SOON”消息只会如果您在作业的 TTR 即将到期时提出保留请求，则会发回。如果您保留一份工作并正在处理它（而不是收集许多其他工作），您将不会看到截止日期消息 - 您没有在寻找它，如果您在寻找它，那么您可能不会首先处理文件。
如果工作线程出现问题并且它不会
触及或删除它所选择的作业，则可以将 TTR 设置视为再次破解作业的第二次机会向上。将超时时间设置为至少与您期望的处理时间一样长，然后添加更多时间作为安全裕度。等待一段时间来执行一项复杂的操作，比一次又一次在同一个操作上失败要好。如果您将 TTR 设置得太低，您将继续获得相同的文件，并且 TTR 将继续过期。According to the mailing list message (part of this thread), the '
DEADLINE_SOON' message will only be sent back if you have a reserve request being made, when a job is about to have the TTR expire on it.If you reserve one job, and are processing it (rather than collecting a number of other jobs), you won't see the deadline message though - you aren't looking for it, and if you were, then you would probably not be processing the file in the first place.
Think of the TTR setting as a second chance to get another crack at a job if there is a problem with the worker and it does not
touchordeletethe job it has picked up. Set the timeout at least as long as you expect the processing to take, and then add a more as a safety margin. Better to wait a while for a complex action than keep failing on the same one over, and over again. If you have set the TTR too low, you'll keep getting the same file, and you'll keep having the TTR expire.