mongoDB:从 pymongo 查询时分组失败

发布于 2024-12-16 13:23:31 字数 4379 浏览 0 评论 0原文

这就是我正在做的事情:

>>> import pymongo
>>> con = pymongo.Connection('localhost',12345)
>>> db = con['staging']
>>> coll = db['contract']
>>> result = coll.group(['asset_id'], None, {'list': []}, 'function(obj, prev) {prev.list.push(obj)}') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.macosx-10.3-fat/egg/pymongo/collection.py", line 908, in group
  File "build/bdist.macosx-10.3-fat/egg/pymongo/database.py", line 340, in command
  File "build/bdist.macosx-10.3-fat/egg/pymongo/helpers.py", line 126, in _check_command_response
pymongo.errors.OperationFailure: command SON([('group', {'$reduce': Code('function(obj, prev) {prev.list.push(obj)}', {}), 'ns': u'contract', 'cond': None, 'key': {'asset_id': 1}, 'initial': {'list': []}})]) failed: exception: BufBuilder grow() > 64MB

我在 mongod 日志上看到的内容如下

Wed Nov 16 16:05:55 [conn209] Assertion: 13548:BufBuilder grow() > 64MB                                                                           0x10008de9b 0x100008d89 0x100151e72 0x100152712 0x100151954 0x100152712 0x100151954 0x100152712 0x100152e7b 0x100152f0c 0x10013b1d9 0x1003706bf 0x10037204c 0x10034c4d6 0x10034d877 0x100180cc4 0x100184649 0x1002b9e89 0x1002c3f18 0x100433888                                                      0   mongod                              0x000000010008de9b _ZN5mongo11msgassertedEiPKc + 315                                                      1   mongod                              0x0000000100008d89 _ZN5mongo10BufBuilder15grow_reallocateEv + 73
 2   mongod                              0x0000000100151e72 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 2962
 3   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 4   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 5   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 6   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 7   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 8   mongod                              0x0000000100152e7b _ZN5mongo9Convertor8toObjectEl + 139
 9   mongod                              0x0000000100152f0c _ZN5mongo7SMScope9getObjectEPKc + 92
 10  mongod                              0x000000010013b1d9 _ZN5mongo11PooledScope9getObjectEPKc + 25
 11  mongod                              0x00000001003706bf _ZN5mongo12GroupCommand5groupESsRKSsRKNS_7BSONObjES3_SsSsPKcS3_SsRSsRNS_14BSONObjBuilderE + 3551
 12  mongod                              0x000000010037204c _ZN5mongo12GroupCommand3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb + 3676
 13  mongod                              0x000000010034c4d6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 1350
 14  mongod                              0x000000010034d877 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 2151
 15  mongod                              0x0000000100180cc4 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 52
 16  mongod                              0x0000000100184649 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 10585
 17  mongod                              0x00000001002b9e89 _ZN5mongo13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 569
 18  mongod                              0x00000001002c3f18 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 1528
 19  mongod                              0x0000000100433888 _ZN5mongo10connThreadEPNS_13MessagingPortE + 616
Wed Nov 16 16:05:55 [conn209] query staging.$cmd ntoreturn:1 command: { group: { $reduce: CodeWScope( function(obj, prev) {prev.list.push(obj)}, {}), ns: "contract", cond: null, key: { asset_id: 1 }, initial: { list: {} } } } reslen:111 1006ms

我对 pymongo 和 Mongodb 都很陌生,不知道如何解决这个问题,请帮助,

谢谢

Here is what I am doing:

>>> import pymongo
>>> con = pymongo.Connection('localhost',12345)
>>> db = con['staging']
>>> coll = db['contract']
>>> result = coll.group(['asset_id'], None, {'list': []}, 'function(obj, prev) {prev.list.push(obj)}') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.macosx-10.3-fat/egg/pymongo/collection.py", line 908, in group
  File "build/bdist.macosx-10.3-fat/egg/pymongo/database.py", line 340, in command
  File "build/bdist.macosx-10.3-fat/egg/pymongo/helpers.py", line 126, in _check_command_response
pymongo.errors.OperationFailure: command SON([('group', {'$reduce': Code('function(obj, prev) {prev.list.push(obj)}', {}), 'ns': u'contract', 'cond': None, 'key': {'asset_id': 1}, 'initial': {'list': []}})]) failed: exception: BufBuilder grow() > 64MB

and what I see on mongod logs is following

Wed Nov 16 16:05:55 [conn209] Assertion: 13548:BufBuilder grow() > 64MB                                                                           0x10008de9b 0x100008d89 0x100151e72 0x100152712 0x100151954 0x100152712 0x100151954 0x100152712 0x100152e7b 0x100152f0c 0x10013b1d9 0x1003706bf 0x10037204c 0x10034c4d6 0x10034d877 0x100180cc4 0x100184649 0x1002b9e89 0x1002c3f18 0x100433888                                                      0   mongod                              0x000000010008de9b _ZN5mongo11msgassertedEiPKc + 315                                                      1   mongod                              0x0000000100008d89 _ZN5mongo10BufBuilder15grow_reallocateEv + 73
 2   mongod                              0x0000000100151e72 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 2962
 3   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 4   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 5   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 6   mongod                              0x0000000100151954 _ZN5mongo9Convertor6appendERNS_14BSONObjBuilderESslNS_8BSONTypeERKNS_13TraverseStackE + 1652
 7   mongod                              0x0000000100152712 _ZN5mongo9Convertor8toObjectEP8JSObjectRKNS_13TraverseStackE + 1682
 8   mongod                              0x0000000100152e7b _ZN5mongo9Convertor8toObjectEl + 139
 9   mongod                              0x0000000100152f0c _ZN5mongo7SMScope9getObjectEPKc + 92
 10  mongod                              0x000000010013b1d9 _ZN5mongo11PooledScope9getObjectEPKc + 25
 11  mongod                              0x00000001003706bf _ZN5mongo12GroupCommand5groupESsRKSsRKNS_7BSONObjES3_SsSsPKcS3_SsRSsRNS_14BSONObjBuilderE + 3551
 12  mongod                              0x000000010037204c _ZN5mongo12GroupCommand3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb + 3676
 13  mongod                              0x000000010034c4d6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 1350
 14  mongod                              0x000000010034d877 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 2151
 15  mongod                              0x0000000100180cc4 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi + 52
 16  mongod                              0x0000000100184649 _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 10585
 17  mongod                              0x00000001002b9e89 _ZN5mongo13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 569
 18  mongod                              0x00000001002c3f18 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 1528
 19  mongod                              0x0000000100433888 _ZN5mongo10connThreadEPNS_13MessagingPortE + 616
Wed Nov 16 16:05:55 [conn209] query staging.$cmd ntoreturn:1 command: { group: { $reduce: CodeWScope( function(obj, prev) {prev.list.push(obj)}, {}), ns: "contract", cond: null, key: { asset_id: 1 }, initial: { list: {} } } } reslen:111 1006ms

I am very new to both pymongo and Mongodb, and dont't know how to resolve this, please help

Thank you

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

单身狗的梦 2024-12-23 13:23:31

堆栈跟踪的相关部分是:

exception: BufBuilder grow() > 64MB

基本上,Mongo 不允许您拥有任何大于 64MB 的文档。有关更多详细信息,请参阅此问题(大小限制已提高到 64MB)然后。)

我不确定你想用这个查询做什么。看起来您想要获取每个 asset_id 的对象列表。然而,您的收藏将会超出容量,因为您永远不会区分组中的对象。尝试将 initial 设置为 {'asset_id': '', 'objects': []} 并将 reduce 设置为 function(obj , 上一个) {上一个.asset_id = obj.asset_id; prev.objects.push(obj) 尽管有更有效的方法来执行此查询。

或者,如果您尝试获取与 ID 匹配的所有文档,请尝试:

coll.find({'asset_id': whatevs})

如果您尝试获取对象的计数,请尝试以下操作:

coll.group(
    ['asset_id'], None, {'asset_id': '', 'count': 0}, 
    'function(obj, prev) {prev.asset_id = obj.asset_id; prev.count += obj.count}'
)

The relevant part of your stacktrace is:

exception: BufBuilder grow() > 64MB

Basically, Mongo doesn't allow you to have any document greater than 64MB. See this question for more details (the size limit has been bumped to 64MB since then.)

I'm not sure what you're trying to do with that query. It sort of looks like you want to get a list of objects for each asset_id. However, your collection is going to grow past capacity because you're never differentiating between objects in your group. Try setting your initial to {'asset_id': '', 'objects': []} and reduce to function(obj, prev) {prev.asset_id = obj.asset_id; prev.objects.push(obj) although there are much more efficient ways of doing this query.

Alternatively, if you're trying to get all the documents matching an ID, try:

coll.find({'asset_id': whatevs})

If you're trying to get a count of the objects, try this instead:

coll.group(
    ['asset_id'], None, {'asset_id': '', 'count': 0}, 
    'function(obj, prev) {prev.asset_id = obj.asset_id; prev.count += obj.count}'
)
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文