后台处理视频上传,PHP有什么熟练的方法?

发布于 2024-11-29 11:28:02 字数 457 浏览 5 评论 0原文

我正在开发一个视频上传网站,但遇到了一个困境:上传的视频需要转换为 FLV 格式才能显示给访问者,但是,如果我在脚本中执行命令,脚本将挂起大约FFMPEG 转换视频需要 10-15 分钟。

我有一个想法,在数据库中插入一条记录,指示需要处理该文件,然后使用设置为每 5 分钟一次的 cron 作业从数据库中选择需要处理的记录,处理它们,然后更新数据库显示它们已被处理。我担心的是执行太多进程并且服务器在压力下崩溃,所以有人对此有任何解决方案或更好的方法来改善我想到的进程吗?


好的,这就是我现在的想法,因此用户上传视频,并向数据库中插入一行,指示需要处理视频。设置为每 5 分钟一次的 cron 作业检查需要处理的内容以及正在处理的内容,假设我一次最多进行五个进程,因此脚本将检查是否需要处理任何视频以及有多少个视频正在处理,如果少于五个,它会更新记录表明它正在处理,一旦视频被处理,它会更新记录表明它已经被处理并且 cron 作业再次启动,有什么想法吗?

I am developing a video upload site and I have ran into a dilemma: videos uploaded need to be converted into the FLV format in order to be displayed to a visitor but, if I execute the command within the script, the script will hang for about 10-15 minutes while the FFMPEG converts the video.

I had an idea to insert a record in to the database indicating the file needs to be processed, then using a cron job set to every 5 minutes to select records from the database which needs to be processed, process them, then update the database showing they have been processed. My worry about this is executing too many processes and the server crashing under the strain, so has anyone got any solutions to this or a way to better the process I have in mind?


Okay, this is now what I have in mind, so the user uploads a video and a row is inserted in to the database indicating the video needs to be processed. A cron job set to every 5 minutes checks what needs to be processed and what is being processed, say I would make a maximum of five processes at one time, so the script would check if any video needs to be processed and how many videos are being processed, if it is less then five, it updates the record indicating that it is being processed, once the video has been processed, it updates the record indicating it has been processed and the cron job starts again, any thoughts?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(6

怀里藏娇 2024-12-06 11:28:02

Gearman 是解决此类问题的一个很好的解决方案,它可以让你立即调度一个工作并拥有任意数量的工人(可能位于不同的服务器上)可用于实现它。

首先,您可以在同一台服务器上运行一些工作程序,但是如果您开始遇到负载问题,那么您可以启动另一台服务器并安装更多工作程序,因此它是水平可扩展的。

Gearman is a good solution for this kind of problem, it lets you instantly dispatch a job and have any number of workers (which may be on different servers) available to fulfill it.

To start with you can run a few workers on the same server, but if you start to run into load issues then you can just fire up another server with some more workers, so it's horizontally scalable.

亚希 2024-12-06 11:28:02

如果您使用 PHP-FPM,则可以使用 PHP.net 上记录的 fastcgi_finish_request()FastCGI 进程管理器 (FPM)

fastcgi_finish_request() - 特殊函数,用于完成请求并刷新所有数据,同时继续执行耗时的操作(视频转换、统计处理等);

如果您不使用 PHP-FPM 或者想要更高级的东西,那么您可以考虑使用队列管理器,例如 Gearman 非常适合您所描述的场景。与使用 shell_exec 运行进程相比,使用 Gearman 的优点是您可以查看正在运行的作业数量/剩余的作业数量并检查它们的状态。您还可以使扩展变得更加容易,因为现在添加作业服务器很简单:

$worker->addServer("10.0.0.1"); 

If you're using PHP-FPM then you can make use of fastcgi_finish_request() as documented on PHP.net. FastCGI Process Manager (FPM)

fastcgi_finish_request() - special function to finish request and flush all data while continuing to do something time-consuming (video converting, stats processing etc.);

If you're not using PHP-FPM or want something more advanced then you might consider using a queue manager like Gearman which is perfectly suited to the scenario you're describing. The advantage of using Gearman over running a process with shell_exec is you can take a look at how many jobs are running / how many are left and check their statuses. You also make scaling much easier as it's now trivial to add job servers:

$worker->addServer("10.0.0.1"); 
硬不硬你别怂 2024-12-06 11:28:02

我喜欢PHP手册中的这个类(参见具体评论): http: //www.php.net/manual/en/function.exec.php#88704

基本上,它可以让您在 *Nix 系统上启动后台进程。它返回一个pid,您可以将其存储在会话中。当您重新加载页面来检查它时,您只需使用保存的 pid 重新创建 ForkedProcess 类,然后就可以检查它的状态。如果已完成,则该过程应该完成。

它不允许进行太多的错误检查,但它非常轻量。

I love this class (see the specific comment) in the PHP manual: http://www.php.net/manual/en/function.exec.php#88704

Basically, it lets you spin off a background process on *Nix systems. it returns a pid, which you can store in the session. When you reload the page to check on it, you simply recreate the ForkedProcess class with the saved pid, and you can check on it's status. If it's complete, the process should be done.

It doesn't allow for much error checking, but it's incredibly lightweight.

雨的味道风的声音 2024-12-06 11:28:02

如果您预计会有大量流量,您应该认真考虑专用服务器。

在单个服务器上,您可以使用 shell_exec 以及 UNIX nohup 命令来获取进程的 PID。

   function run_in_background($Command, $Priority = 0)
   {
       if($Priority)
           $PID = shell_exec("nohup nice -n $Priority $Command 2> /dev/null & echo $!");
       else
           $PID = shell_exec("nohup $Command 2> /dev/null & echo $!");
       return($PID);
   }
   function is_process_running($PID)
   {
       exec("ps $PID", $ProcessState);
       return(count($ProcessState) >= 2);
   }

该技术的完整描述如下: http ://nsaunders.wordpress.com/2007/01/12/running-a-background-process-in-php/

您也许可以将 PID 列表放入 MySQL 表中,然后使用您的每 5 分钟执行一次 cron 作业来检测视频何时完成并更新数据库中的相关值。

If you expect a lot of traffic you should seriously consider a dedicated server.

On a single server, you can use shell_exec along with the UNIX nohup command to get the PID of the process.

   function run_in_background($Command, $Priority = 0)
   {
       if($Priority)
           $PID = shell_exec("nohup nice -n $Priority $Command 2> /dev/null & echo $!");
       else
           $PID = shell_exec("nohup $Command 2> /dev/null & echo $!");
       return($PID);
   }
   function is_process_running($PID)
   {
       exec("ps $PID", $ProcessState);
       return(count($ProcessState) >= 2);
   }

A full description of this technique is here: http://nsaunders.wordpress.com/2007/01/12/running-a-background-process-in-php/

You could perhaps put the list of PIDs in a MySQL table and then use your cron job every 5 mins to detect when a video is complete and update the relevant values in the database.

情深缘浅 2024-12-06 11:28:02

您可以使用 system 调用 ffmpeg 并将输出发送到 /dev/null,这将使该调用立即返回,从而在后台有效地处理它。

You can call ffmpeg use system and send the output to /dev/null, this will make that call return right away, effectively handling it in the background.

迟月 2024-12-06 11:28:02

生成几个工作进程,它们将使用消息队列中的消息,例如 beanstalkd。通过这种方式,您可以控制并发任务(转换)的数量,并且不必付出生成进程的代价(因为进程保持在后台运行)。

我认为如果您使用/编码 C 并使用 Redis 作为消息队列,速度会快得多。 Redis 有一个非常好的 C 客户端库,名为 Hiredis。我不认为这会非常难以实现。

Spawn couple of worker processes which will consume messages from message queue like for example beanstalkd. This way you can control the number of concurrent tasks(conversions) and also don't have to pay price of spawning processes(because processes keep running in background).

I think it would be even a lot faster if you used/coded C and used Redis as your message queue. Redis has a very good c client library named Hiredis. I don't think this would be insanely difficult to accomplish.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文