MongoView Creation 给出的排序超出了 33554432 字节的内存限制,但没有选择外部排序

发布于 2025-01-17 21:07:07 字数 3550 浏览 1 评论 0原文

我的mongo聚合查询给出以下错误

stacktrace: 数据库错误! | ____/ mongo服务器错误(mongoqueryException):错误代码292和错误消息的查询失败,并且在聚合过程中planexecutor错误::由:: codion :: stort ::排序超过33554432字节的内存限制,但没有选择在外部排序。''在下面的

上是我的观点聚合管道


[{
    $sort: {
        event_trigger_date: -1
    }
}, {
    $group: {
        _id: {
            "profile_id": "$profile_id"
        },
        data: {
            $first: "$$ROOT"
        }
    }
}, {
    $unset: "_id"
}, {
    $replaceRoot: {
        newRoot: "$data"
    }
}, {
    $project: {
        "profile_id": 1
    }
}, {
    $lookup: {
        from: 'profile_event',
        localField: 'profile_id',
        foreignField: 'profile_id',
        as: 'profile_event_data'
    }
}, {
    $group: {
        _id: {
            "profile_id": "$profile_id"
        },
        data: {
            $first: "$$ROOT"
        }
    }
}, {
    $replaceRoot: {
        newRoot: "$data"
    }
}, {
    $project: {
        profile_id: 1,
        profile_event_data: 1,
        event_type_set: {
            $concatArrays: ["$profile_event_data.event_type"]
        }
    }
}, {
    $addFields: {
        _id: {
            $concat: ["ACTIONS_NOT_COMPLETED_0X:", "$profile_id"]
        },
        event_type: "ACTIONS_NOT_COMPLETED_NX",
        event_trigger_date: "$$NOW",
        event_occurence: 0,
        trigger_status: "SILENT"
    }
}, {
    $unset: "event_exists"
}, {
    $lookup: {
        from: 'profile_personal_info',
        localField: 'profile_id',
        foreignField: 'profile_id',
        as: 'personal_info'
    }
}, {
    $project: {
        profile_id: 1,
        event_type: 1,
        event_trigger_date: 1,
        event_occurence: 1,
        trigger_status: 1,
        event_type_set: 1,
        personal_info: {
            $arrayElemAt: ["$personal_info", 0]
        }
    }
}, {
    $addFields: {
        oldest_personal_info_created_date: {
            $trunc: {
                $divide: [{
                    $subtract: ["$$NOW", '$personal_info.created_date']
                }, 1000 * 60 * 60 * 24]
            }
        }
    }
}, {
    $addFields: {
        created_date: {
            $trunc: {
                $divide: [{
                    $subtract: ["$$NOW", '$event_trigger_date']
                }, 1000 * 60 * 60 * 24]
            }
        }
    }
}, {
    $project: {
        event_type: 1,
        profile_id: 1,
        event_trigger_date: 1,
        profile_event_data: 1,
        event_type_set: 1,
        event_occurence: 1,
        trigger_status: 1,
        category_value: {
            $cond: {
                if: {
                    $eq: ["$oldest_personal_info_created_date", null]
                },
                then: "$created_date",
                else: "$oldest_personal_info_created_date"
            }
        }
    }
}, {
    $project: {
        profile_id: 1,
        event_type: 1,
        event_type_set: 1,
        event_trigger_date: 1,
        event_occurence: 1,
        trigger_status: 1,
        category_value: 1,
        "event_exists": {
            $in: ["ACTIONS_NOT_COMPLETED_NX", "$event_type_set"]
        }
    }
}, {
    $match: {
        event_exists: {
            $ne: true
        }
    }
}, {
    $unset: ["event_exists", "event_type_set"]
}]

文章建议的问题是因为逻辑消耗了100 MB的RAM空间,因此我从聚合管道中删除了排序,但它给了我以下错误

stacktrace: | ____/ mongo服务器错误(mongoqueryException):查询错误代码4568和错误消息在汇总过程中的planexecutor错误::由:: cody profy :: profile_event匹配pipeline的$查找级超过16793600字节中的文档总数超过16793600字节>

My Mongo Aggregation query gives following errors

Stacktrace:
Database error!
|____/ Mongo Server error (MongoQueryException): Query failed with error code 292 and error message 'PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting.' on

Below is my view aggregation pipeline


[{
    $sort: {
        event_trigger_date: -1
    }
}, {
    $group: {
        _id: {
            "profile_id": "$profile_id"
        },
        data: {
            $first: "$ROOT"
        }
    }
}, {
    $unset: "_id"
}, {
    $replaceRoot: {
        newRoot: "$data"
    }
}, {
    $project: {
        "profile_id": 1
    }
}, {
    $lookup: {
        from: 'profile_event',
        localField: 'profile_id',
        foreignField: 'profile_id',
        as: 'profile_event_data'
    }
}, {
    $group: {
        _id: {
            "profile_id": "$profile_id"
        },
        data: {
            $first: "$ROOT"
        }
    }
}, {
    $replaceRoot: {
        newRoot: "$data"
    }
}, {
    $project: {
        profile_id: 1,
        profile_event_data: 1,
        event_type_set: {
            $concatArrays: ["$profile_event_data.event_type"]
        }
    }
}, {
    $addFields: {
        _id: {
            $concat: ["ACTIONS_NOT_COMPLETED_0X:", "$profile_id"]
        },
        event_type: "ACTIONS_NOT_COMPLETED_NX",
        event_trigger_date: "$NOW",
        event_occurence: 0,
        trigger_status: "SILENT"
    }
}, {
    $unset: "event_exists"
}, {
    $lookup: {
        from: 'profile_personal_info',
        localField: 'profile_id',
        foreignField: 'profile_id',
        as: 'personal_info'
    }
}, {
    $project: {
        profile_id: 1,
        event_type: 1,
        event_trigger_date: 1,
        event_occurence: 1,
        trigger_status: 1,
        event_type_set: 1,
        personal_info: {
            $arrayElemAt: ["$personal_info", 0]
        }
    }
}, {
    $addFields: {
        oldest_personal_info_created_date: {
            $trunc: {
                $divide: [{
                    $subtract: ["$NOW", '$personal_info.created_date']
                }, 1000 * 60 * 60 * 24]
            }
        }
    }
}, {
    $addFields: {
        created_date: {
            $trunc: {
                $divide: [{
                    $subtract: ["$NOW", '$event_trigger_date']
                }, 1000 * 60 * 60 * 24]
            }
        }
    }
}, {
    $project: {
        event_type: 1,
        profile_id: 1,
        event_trigger_date: 1,
        profile_event_data: 1,
        event_type_set: 1,
        event_occurence: 1,
        trigger_status: 1,
        category_value: {
            $cond: {
                if: {
                    $eq: ["$oldest_personal_info_created_date", null]
                },
                then: "$created_date",
                else: "$oldest_personal_info_created_date"
            }
        }
    }
}, {
    $project: {
        profile_id: 1,
        event_type: 1,
        event_type_set: 1,
        event_trigger_date: 1,
        event_occurence: 1,
        trigger_status: 1,
        category_value: 1,
        "event_exists": {
            $in: ["ACTIONS_NOT_COMPLETED_NX", "$event_type_set"]
        }
    }
}, {
    $match: {
        event_exists: {
            $ne: true
        }
    }
}, {
    $unset: ["event_exists", "event_type_set"]
}]

Articles suggested issue is because of sort logic consuming more than 100 Mb of RAM Space so i removed sort from my aggregation pipeline but it gave me following error

Stacktrace:
|____/ Mongo Server error (MongoQueryException): Query failed with error code 4568 and error message 'PlanExecutor error during aggregation :: caused by :: Total size of documents in profile_event matching pipeline's $lookup stage exceeds 16793600 bytes' on server

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文