用于长时间运行作业的 PHP Cron Job 的替代方案
我们有一个网站，其中实时列出博客链接。问题是页面加载速度很慢，因为它们正在从各个源站点读取数据。
我编写了一个 PHP 脚本，用于创建每个页面的 HTML 版本。该操作每小时运行一次。问题是 PHP 脚本在完成所有页面之前就超时了。我知道我可以增加 PHP 脚本允许的执行时间，但这似乎不是处理该问题的最有效方法。
还有其他方法可以做到这一点吗？我只是不知道该开始寻找什么 - PERL？ JAVA？ Python？这些脚本如何在服务器上运行？我应该从我的网络托管服务商那里寻找什么？
We have a website that lists links to blogs in realtime. The problem is that the pages are slow to load because they are reading data from the various source sites.
I wrote a PHP script that creates an HTML version of each page. This runs once each hour. The problem is that the PHP script is timing out before it finishes all the pages. I know that I could increase the execute time allowed for PHP scripts, but this does not seem like the most efficient way to handle the issue.
Is there another way to do this? I just don't know what to begin looking for - PERL? JAVA? Python? How do these scripts run on a server? What should I look for from my web host?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
使用 urllib2 的 Python 可能会做得很好。另外，我的理解是否正确：您有一个聚合来自其他站点的数据的站点，并且它都是生成的静态 HTML？听起来您好像正在使用 HTML 作为数据库，因此也许可以考虑使用合适的数据库。
Python with urllib2 will probably do a good job. Also, do I understand this right: you have a site that aggregates data from other sites, and it's all generated static HTML? It sounds like you're kind of using HTML as a database, so maybe consider using a proper one.
鉴于您最初的问题是网络延迟之一（“页面加载缓慢”），我认为没有理由相信 PHP 是这里的瓶颈。我怀疑更改语言会影响您的脚本运行时间。
Given the fact that your original problem is one of network latency ("pages are slow to load") I see no reason to believe that PHP is the bottleneck here. I doubt changing languages will affect your script run time.
一个不同的解决方案可能是使用数据库，而不是一次完成这么多工作。制作一个表格，列出您拉取的网站，并存储上次拉取的时间。然后让 cron 拉出 1 或 2 个有一段时间没有拉出的。让它经常运行，那么您将始终拥有新鲜数据，但脚本将更容易工作，因为它不会尝试一次执行这么多操作。这个概念将会很好地扩展。
A different solution might be to use a database, and not bite off so much work at once. Make a table listing the sites you pull, and store when they were last pulled. Then have the cron pull out 1 or 2 that haven't been pulled in a while. Have it run often, then you'll always have fresh data, but the script will have an easier time working as its not trying to do so much at once. This concept will scale well.