Mongo在数据查找时找不到master

发布于 2024-10-29 15:41:21 字数 603 浏览 2 评论 0原文

我正在使用 pymongo 运行大型数据更新。要运行更新,需要使用collection.find_one(unique criteria)找到各个记录,进行更改,批量更新,最后使用db.collection.save([要保存的长记录列表])

在我的本地计算机(运行 1.6.3)上,导入工作正常。

在比我的本地计算机快得多的远程服务器(运行1.6.0)上,我可以很好地完成部分插入,但在查找原始记录时会突然出现以下错误:

connection = Connection(...)
...
raise AutoReconnect("could not find master/primary")
pymongo.errors.AutoReconnect: could not find master/primary

记录数我能通过的情况有所不同,但不是随机的。

起初我以为我遇到了连接限制。我在每次记录查找后开始手动关闭连接:

collection.database.connection.disconnect()

这并没有解决问题。我走在正确的轨道上吗?

I am running a large data update using pymongo. To run the updates, individual records are found using collection.find_one(unique criteria), changes are made, the updates are batched, and finally sent in chunks using db.collection.save([long list of records to save])

On my local machine (running 1.6.3), the imports work fine.

On a remote server (running 1.6.0), which is much faster than my local machine, I can get through a portion of the inserts just fine, but then will suddenly get the following error when looking up original records:

connection = Connection(...)
...
raise AutoReconnect("could not find master/primary")
pymongo.errors.AutoReconnect: could not find master/primary

The number of records I can get through is varies somewhat, but is not random.

At first I thought I was running into the connection limit. I started closing connections manually after each record lookup:

collection.database.connection.disconnect()

Which didn't solve the problem. Am I on the right track?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

深海不蓝 2024-11-05 15:41:21

因此,这里存在一些潜在的问题:

raise AutoReconnect("could not find master/primary") 
pymongo.errors.AutoReconnect: could not find master/primary

该错误表明现有连接已以某种方式失效。发生这种情况的原因有很多。

发生这种情况的最常见原因是副本集的主节点已下台或发生故障。在这种情况下,您的代码需要:

  1. 捕获(或捕获)错误。
  2. 决定重试策略。 (失败?重试一次?...)

你在这样做吗?
您运行的是副本集还是主/从?
您对这些服务器的性能有任何跟踪吗?
他们有网络问题吗?
他们在互换角色吗?

collection.database.connection.disconnect()

这并没有解决问题。我走在正确的道路上吗?

异常“发生”在哪里?它来自连接本身还是保存命令?

在远程服务器上(运行 1.6.0)

在撰写本文时,1.6.0 是 MongoDB 的一个非常旧的版本。在后续的 1.6.x 版本和 1.7.x 版本中修复了多个复制错误。 (我们已经是 1.8.1rc-0)

我会首先查看您的服务器发生的情况,但这很可能会引导您走上升级之路。

So there are a couple of potential issues here:

raise AutoReconnect("could not find master/primary") 
pymongo.errors.AutoReconnect: could not find master/primary

That error indicates that the existing connection has somehow been invalidated. There are a number of reasons this could happen.

The most common reason this happens is that that the Primary of a Replica Set has stepped down or has failed. In this case your code needs to:

  1. Catch (or trap) the error.
  2. Decide on a retry strategy. (fail? retry once?...)

Are you doing this?
Are you running Replica Sets or Master/Slave?
Do you have any tracking for the performance of these servers?
Are they having network issues?
Are they switching roles?

collection.database.connection.disconnect()

Which didn't solve the problem. Am I on the right track?

Where is the exception "happening"? Is it coming from the connection itself or the save command?

On a remote server (running 1.6.0)

As of this writing, 1.6.0 is a very old version of MongoDB. There were multiple replication bugs fixed in the subsequent 1.6.x versions and 1.7.x versions. (we're already at 1.8.1rc-0)

I would start by looking at what's happening with your servers, but that may well lead you down the upgrade path.

安穩 2024-11-05 15:41:21

我在与 pymongo 的交互式 python 使用中遇到了这个问题,我让会话空闲并在返回时遇到 AutoReconnect 。我是这样处理的:

import functools
import pymongo
import time

MAX_AUTO_RECONNECT_ATTEMPTS = 5

def graceful_auto_reconnect(mongo_op_func):
  """Gracefully handle a reconnection event."""
  @functools.wraps(mongo_op_func)
  def wrapper(*args, **kwargs):
    for attempt in xrange(MAX_AUTO_RECONNECT_ATTEMPTS):
      try:
        return mongo_op_func(*args, **kwargs)
      except pymongo.errors.AutoReconnect as e:
        wait_t = 0.5 * pow(2, attempt) # exponential back off
        time.sleep(wait_t)

  return wrapper

@graceful_auto_reconnect
def some_func_that_does_mongodb_ops():
  ...
  ...

YMMV

I've encountered this problem in interactive python usage with pymongo, where I leave the session idle and encounter AutoReconnect upon returning. I've handled it this way:

import functools
import pymongo
import time

MAX_AUTO_RECONNECT_ATTEMPTS = 5

def graceful_auto_reconnect(mongo_op_func):
  """Gracefully handle a reconnection event."""
  @functools.wraps(mongo_op_func)
  def wrapper(*args, **kwargs):
    for attempt in xrange(MAX_AUTO_RECONNECT_ATTEMPTS):
      try:
        return mongo_op_func(*args, **kwargs)
      except pymongo.errors.AutoReconnect as e:
        wait_t = 0.5 * pow(2, attempt) # exponential back off
        time.sleep(wait_t)

  return wrapper

@graceful_auto_reconnect
def some_func_that_does_mongodb_ops():
  ...
  ...

YMMV

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文