PHP FTP 上传数千个文件

发布于 2024-08-27 18:50:37 字数 203 浏览 8 评论 0原文

我编写了一个小型 FTP 类,用于将文件从本地服务器移动到远程服务器。它通过检查本地文件数组与远程服务器上的文件数组来实现此目的。如果该文件存在于远程服务器上,则不会上传它。

该脚本对于少量文件工作正常,但我注意到本地服务器可以传输多达 3000 个以上的图像文件,这似乎会导致脚本失败并且仅传输 100 个左右。

如何修改脚本以处理潜在的数千个图像传输文件?

I've written a small FTP class which I used to move files from a local server to a remote server. It does this by checking an array of local files with an array of files on the remote server. If the file exists on the remote server, it won't bother uploading it.

The script works fine for small amounts of files, but I've noticed that the local server can have as many as 3000+ image files to transfer, this seems to cause the script to flop and only transfer a 100 or so.

How can I modify the script to handle potentially thousands of image transfer files?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

じее 2024-09-03 18:50:37

更频繁地运行 cron,并将脚本限制为每次运行上传 80 个图像。

Run cron more frequently, and limit the script to an upload of 80 images per run.

月棠 2024-09-03 18:50:37

可能发生的情况是您执行脚本的时间过长(这不适用于命令行 php),如果发生这种情况,您的脚本将被 Web 服务器停止。您可以更改 php 设置来解决该问题,但这不会很好地扩展(因为您的浏览器最终也会超时)。也许从命令行(称为 cli php)运行脚本会起作用。

在我看来,你正在实施一些已经存在的东西。如果您可以控制两台服务器,您应该看看rsync(对于Linux)。

What could happen is that you take too long to execute the script (this doesn't apply to commandline php) if that happens your script will be stopped by the web server. You can change php settings to fix that, but that will not scale very well (because your browser will eventually time out too). Perhaps running the script from commandline (called cli php) will work.

It sounds to me like you are implementing something that already exists. You should have a look at rsync (for linux) if you have control of both servers.

小草泠泠 2024-09-03 18:50:37

如果问题出在 php 或浏览器超时,您可以创建一个文件(下面的示例)并对其进行 cron 操作或从浏览器调用。

<?
echo "Running cli syncfiles.php";
system("&php syncfiles.php"); // & pushes file to background processing on linux 
?>

如果您遇到问题是因为 ftp 限制您的连接,或者限制您在 x 时间内同时上传/下载,那么您可以在代码中添加某种计时器。

<?
$counter=0;
for($i=0;$i<$numftpfiles;$i++)
{
   syncfile($i); // this represents your sync code
   usleep(250000); // sleep for 1/4 second
   $count++;
   if($count>50)
   {
     usleep(30000000); // sleep for 30 seconds
     $count=0;
   }
}
?>

If the problem is with php or browser time out you can create a file (example below) and cron it or call from a browser.

<?
echo "Running cli syncfiles.php";
system("&php syncfiles.php"); // & pushes file to background processing on linux 
?>

If you are having a problem because the ftp is throttling your connections, or is throttling your simultaneous uploads/downloads within x amount of time, then you can probably throw some kind of timers into the code.

<?
$counter=0;
for($i=0;$i<$numftpfiles;$i++)
{
   syncfile($i); // this represents your sync code
   usleep(250000); // sleep for 1/4 second
   $count++;
   if($count>50)
   {
     usleep(30000000); // sleep for 30 seconds
     $count=0;
   }
}
?>
我也只是我 2024-09-03 18:50:37

你可以先在 php 中压缩
http://www.php.net/manual/en/book.zip。 php

然后上传一个较大的 zip 文件。总文件大小不太可能改变,但我发现当通过 WAN 传输大量文件时,速度会更快。

-将要

You could zip then first in php
http://www.php.net/manual/en/book.zip.php

Then upload one larger zip file. The total file size is unlikely to change, but I've found when doing transfers of a lot of files across my WAN it is faster anyway.

-Will

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文