如何避免异步进程的等待时间?

发布于 2024-12-03 11:18:24 字数 1808 浏览 0 评论 0原文

我有一个简单的报告脚本,需要 5-10 分钟才能运行。它通过 fsockopen 调用异步触发并在后台运行。有时它的效果非常好。但是,如果用户在上一个页面加载触发的异步报告仍在运行时刷新主页,则主页将挂起,直到第一个页面加载的异步过程完成。

下面是我的脚本背后的粗略逻辑,然后是有关哪些有效、哪些无效的更多详细信息...


ma​​in.php

if last report was already run within the hour (mysql select last report time)
    display existing report, that's it.
else
    log latest report process request
    run report asynchronously in the background (async.php)
    notification when updated report is completed (simple ajax pinger every 10s)

async.php

run report
update last report request table with "complete" status
the ajax pinger in main.php pulls the "complete" record and triggers notification

问题< /strong>

  1. 用户加载main.php

    结果: 太棒了。立即加载,记录新的报告请求并按预期触发异步调用。 Async.php 现在需要 5-10 分钟才能完成报告。一切都在后台进行。

  2. 用户在 20 分钟后加载 main.php(async.php 完成后)

    结果: 很好,立即加载,但会跳过运行异步报告进程,因为它距离上次请求不到一小时。

  3. 用户等待一个小时并加载 main.php再次

    结果: 一切顺利,如步骤 1

    到目前为止一切都很好,但是...

  4. 现在用户在步骤 3 后仅 2 分钟就加载了 main.php。

    结果: 失败! 此页面加载将挂起,直到步骤 3 中触发的异步进程完成。即使带有请求日志时间的 mysql 表在步骤 3 中立即更新。因此步骤 4 应该简单地跳过调用并像步骤 2 一样渲染现有报告。

到底是什么?你建议什么来调试这个?如果当用户 1 挂在步骤 4 上时另一个用户想要运行相同的报告,则该其他用户可以运行一个新报告。

我的代码中没有做任何花哨的事情。只是简单的 if/then 和 mysql 选择查找。异步脚本主要从外部源获取数据,因此它不会锁定 mysql 表,这可能会阻止请求日志时间查找(完整的报告可能仅在 10 分钟内运行一百个 5 毫秒的查询,并且没有一个会触及请求日志)。

一种解决方案是仅对进程进行 cron 操作,但我担心我正在运行大量永远不会被看到的报告。如果我更频繁地运行它们,报告就会成倍增长,而不是只需要 5-10 分钟。

那么,坚持我的上述攻击计划(目前),您有什么建议?为什么步骤 2 可以正确加载,但步骤 4 却不能正确加载?是否存在我不知道的某种脚本锁定或每个用户限制?

I have a simple reporting script that takes 5-10 minutes to run. It's triggered asynchronously via an fsockopen call to run in the background. It works wonderfully some of the time. But if a user refreshes the main page while the async report triggered from the previous pageload is still running the main page hangs until the async process from the first page load completes.

Below is the rough logic behind my scripts, followed by more details of what works and what doesn't...


main.php

if last report was already run within the hour (mysql select last report time)
    display existing report, that's it.
else
    log latest report process request
    run report asynchronously in the background (async.php)
    notification when updated report is completed (simple ajax pinger every 10s)

async.php

run report
update last report request table with "complete" status
the ajax pinger in main.php pulls the "complete" record and triggers notification

The problem

  1. User loads main.php

    Result: Great. Loads instantly, logs new report request and triggers asynchronous call as expected. Async.php will now take 5-10 minutes to complete report. All in the background.

  2. User loads main.php 20 minutes later (after the async.php finishes)

    Result: Great, loads instantly, but skips running the async report process because it's within an hour of last request.

  3. User waits an hour and loads main.php again

    Result: All good, as in step 1

    All good so far, but...

  4. Now user loads main.php only 2 minutes after step 3.

    Result: FAIL! This page load will hang until the async process triggered in step 3 completes. Even though the mysql table with the request log time is updated instantly in step 3. Therefore step 4 should simply skip the call and just render the existing report like step 2.

What in the world? What do you recommend to debug this? If another user wants to run the same report while user 1 is hanging on step 4, this other user runs a new report just fine.

I'm not doing anything fancy in my code. Just simple if/then and mysql select lookups. The async script largely pulls in from external sources, so it's not locking up mysql tables that might prevent a request log time lookup (a full report may only run a hundred 5ms queries over 10 minutes and none of which touch the request log).

One solution is to just cron the processes, but I'm concerned that I'm running a lot of reports that will never be seen. And instead of a report only taking 5-10 minutes it would get exponentially larger if I ran them all more frequently.

So, sticking with my above plan of attack (for now), what do you recommend? Why would step 2 load correctly, but not step 4? Is there some sort of script lock or limit per user I'm not aware of?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

烂人 2024-12-10 11:18:24

问题是在生成报告时出现的,对吧?

如果是这样,您可以在报告开始生成时创建一个文件,并在报告结束时将其删除,并在启动新一代报告之前检查该文件是否存在。在你的 async.php 文件中:

check if 'running.txt' file exists
if it exists:
    display 'report is already running, you need to wait ! '
    stop
else : 
    create file 'running.txt'
    run report
    delete file 'running.txt'
    update last report request table with "complete" status
    the ajax pinger in main.php pulls the "complete" record and triggers notification

The problem occurs when the report is being generated, right ?

If so, you can just create a file when your report starts generation and delete it when it ends, and check if it exists before launching a new generation of the report. in your async.php file :

check if 'running.txt' file exists
if it exists:
    display 'report is already running, you need to wait ! '
    stop
else : 
    create file 'running.txt'
    run report
    delete file 'running.txt'
    update last report request table with "complete" status
    the ajax pinger in main.php pulls the "complete" record and triggers notification
雪花飘飘的天空 2024-12-10 11:18:24

我最终接受了雷吉莱罗的建议。我设置了一个 cron 每分钟运行一次并接收队列中的任何新请求。这完全消除了滞后问题。谢谢雷吉莱罗!

I ended up taking regilero's advice. I set up a cron to run every minute and pick up any new requests in the queue. This eliminated the lag issue entirely. Thanks regilero!

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文