MongoDB - PHP - MongoCursorException“找不到游标”

发布于 2024-11-25 17:54:49 字数 828 浏览 4 评论 0原文

我有 2 个集合:A(380 万个文档)和 B(170 万个文档)

我有一个从 shell 运行的 PHP 脚本:

  1. 大约 60% 的时间循环遍历 A 中的每条记录
  2. ,它在 B 上执行 findOne (使用_id)
  3. 创建一个php数组

做了一些基本的数学运算,一旦完成a中所有文档的循环,就

: 4)循环php数组

5)在(1)期间将其插入到集合C中

,我始终得到:PHP Fatal错误:未捕获异常“MongoCursorException”,消息“未找到光标” 最后处理的项目是 3872494 中的#8187。

real    1m25.478s
user    0m0.076s
sys     0m0.064s

再次运行它,代码没有更改,异常在项目#19826 / 3872495 处抛出

real    3m19.144s
user    0m0.120s
sys     0m0.072s

,再次,#8181 / 387249

real    1m31.110s
user    0m0.036s
sys     0m0.048s

是的,我意识到我可以(并且可能应该)捕获例外...但是...为什么它会被抛出?尤其是在数据库中经过的时间/深度如此不同的情况下。

如果有帮助的话,我的设置是 3 节点副本集 (2+arb)。我将辅助设备脱机并尝试仅运行主要设备。结果相同(处理的结果数量和次数不同,但总是抛出 Cursor Not Found 异常)。

I have 2 collections: A (3.8M docs) and B (1.7M docs)

I have a PHP script that I run from the shell that:

  1. loops over each record in A
  2. ~60% of the time, it does a findOne on B (using _id)
  3. does some basic math, creating a php array

once the loop on all docs in a is done:

4) loop over php array

5)upsert into collection C

during (1), I consistently get: PHP Fatal error: Uncaught exception 'MongoCursorException' with message 'Cursor not found'
The last item processed was #8187 of 3872494.

real    1m25.478s
user    0m0.076s
sys     0m0.064s

Running it again, with no change in code, the exception got thrown at item #19826 / 3872495

real    3m19.144s
user    0m0.120s
sys     0m0.072s

And again, #8181 / 387249

real    1m31.110s
user    0m0.036s
sys     0m0.048s

Yes, I realize that I can (and probably should) catch the exception... but... why is it even being thrown? Especially at such different elapsed time/depth into the database.

If it helps, my setup is a 3-node replica set (2+arb). I took the secondary offline and tried with just the primary running. Same results (different number of results processed and times, but always throws the Cursor Not Found exception).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

九公里浅绿 2024-12-02 17:54:49

是的,我意识到我可以(并且可能应该)捕获异常...

是的,这绝对是要做的第一件事。异常发生有几十个合理的理由吗?哎呀,当主数据库离线并且无法访问时,您认为会发生什么?

...为什么它会被抛出?

有几个潜在的原因,但让我们直接讨论您看到的错误代码。

  • 官方 PHP 文档位于此处
  • 引自该页面: 驱动程序试图从数据库中获取更多结果,但数据库没有查询记录。这通常意味着游标在服务器端超时...

MongoDB PHP 驱动程序有两种不同的超时:

  • 连接超时
  • 游标超时

您遇到了游标超时。您可以连接到数据库,但您的查询“没有时间”。

可能的修复:

  1. 延长光标超时。或者您可以将其设置为零并使其永久有效。
  2. 分批进行这项工作。从 A 获取前 1000 个 _ids,处理它们,然后标记您已完成此操作。然后获取比上次运行大的下 1000 个 _ids,依此类推。

我建议#2 以及处理异常。即使这些不能完全解决问题,它也会帮助您隔离和缓解问题。

Yes, I realize that I can (and probably should) catch the exception...

Yes, this is definitely the first thing to do. There are dozens of legitimate reasons for an exception to happen? Heck what do you think happens when the primary goes off-line and becomes unreachable?

... why is it even being thrown?

There are a couple of potential reasons, but let's cut straight to the error code you're seeing.

  • Official PHP docs are here.
  • Quote from that page: The driver was trying to fetch more results from the database, but the database did not have a record of the query. This usually means that the cursor timed out on the server side...

The MongoDB PHP driver has two different timeouts:

  • Connection timeout
  • Cursor timeout

You're hitting a cursor timeout. You can connect to the DB, but your query is "running out of time".

Possible fixes:

  1. Extend the cursor timeout. Or you can set it to zero and make it last forever.
  2. Do this job in batches. Get the first 1000 _ids from A, process them and then mark that you have done so. Then get the next 1000 _ids greater than your last run and so on.

I would suggest #2 along with handling the exception. Even if these doesn't completely solve the problem it will help you isolate and mitigate the problem.

抱着落日 2024-12-02 17:54:49

我知道已经晚了,这可能不是您的解决方案,但您可以尝试使用 immortal()。正如盖茨副总裁指出的那样,此页面描述了该异常。

驱动程序试图从数据库获取更多结果,但数据库没有查询记录。这通常意味着游标在服务器端超时:几分钟不活动后,数据库将终止游标(请参阅 MongoCursor::immortal() 了解有关防止这种情况的信息)。

我想我应该为其他到达此页面的人发布完整的描述,因为 timeout() 和 immortal() 是不同的。 timeout() 设置等待响应的时间。 immortal() 拒绝光标因不活动而死亡。

I know it's late and this may not be your solution but you can try using immortal(). As Gates VP noted, this page describes the exception.

The driver was trying to fetch more results from the database, but the database did not have a record of the query. This usually means that the cursor timed out on the server side: after a few minutes of inactivity, the database will kill a cursor (see MongoCursor::immortal() for information on preventing this).

I figured I'd post the entire description for others reaching this page and since timeout() and immortal() are different. timeout() sets the amount of time to wait for a response. immortal() denies the cursor from dying due to inactivity.

定格我的天空 2024-12-02 17:54:49

这可能是内存限制问题。尝试提供更多内存,看看结果是否有所不同,您可以使用 -d 选项来做到这一点: php -d memory_limit=256M yourscript.php

这是很多文档,听起来您正在制作一个相当大的文档对象数组。还有各种 php 函数,例如 memory_get_usage(),您可以使用它们在运行时分析内存分配以及调试扩展,例如 xdebug 或 zend 提供的功能。

It could be a memory limit issue. Try experimenting with providing more memory and see if your results vary, which you can do with the -d option: php -d memory_limit=256M yourscript.php

That is a lot of documents, and it sounds like you're making a pretty large array of objects. There are also various php functions like memory_get_usage() you can use to profile your memory allocation at runtime as well as debugging extensions like xdebug or what zend provides.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文