对于相同的查询,官方 Javascript MongoDB 驱动程序比 Python PyMongo 慢?

发布于 2025-01-14 04:48:54 字数 2427 浏览 0 评论 0原文

我一直在为我的新后端尝试 Javascript,但我注意到一些奇怪的事情。当我使用 Python 的 PyMongo 库时,获取所有数据的运行速度是我使用官方 Javascript MongoDB 模块时的两倍(33.5 秒 -> 16.44 秒)。测试设置如下:

mongo = MongoClient(URI) # initializing client

rn = time() # starting timer
for ID in LIST_OF_IDS:
    results = mongo["current_day_info"][ID].find() # fetches all the documents in the collection named <ID>
    results = [[result["time"].timestamp() / 10, result["buy_price"], result["sell_price"], result["buy_volume"], result["sell_volume"], result["week_buy_amount"], result["week_sell_amount"]] for result in results] # expands list of documents into array of their values
print("mongodb", time()-rn) # ending timer
const client = new MongoClient(URI); // initializing client

async function main() {
    await client.connect(); // connecting to client
    const database = client.db("current_day_info"); // goes to proper database

    console.time("mongodb"); // starting timer
    for (let ID of LIST_OF_IDS) {
        let results = [];
        const documents = database.collection(ID).find({}); // fetches all the documents in the collection named <ID>
        documents.forEach(item => results.push([new Date(item.time).getTime(), item.buy_price, item.sell_price, item.buy_volume, item.sell_volume, item.week_buy_amount, item.week_sell_amount])); // expands list of documents into array of their values
    }
    console.timeEnd("mongodb"); // ending timer
}

main();

我最好的猜测是 PyMongo 有一些部分是用 C 编写的,但我不认为这会导致如此大幅的增长?

版本: Python 3.8.10、PyMongo 4.0.2、MongoDB 版本 5.0.6、MongoDB 节点驱动程序版本 4.4.1、NodeJS 版本 17.7.1


从下面的评论复制:

示例文档:

{
   "time":"ISODate(""2022-03-12T23:23:45.000Z"")",
   "_id":"ObjectId(""622d2b8c83d4c06e792895cb"")",
   "buy_volume":2079625,
   "sell_price":5.5,
   "sell_volume":10210328,
   "week_sell_amount":68184205,
   "week_buy_amount":10783950,
   "buy_price":7.4
}

另外,在 MongoSH 中尝试了同样的操作,但花费的时间明显更长(7 分钟),我想我在那里弄乱了一些东西:

async function main() { 
    console.time("mongodb");
    for (let ID of LIST_OF_IDS) {
        let results = [];
        const collection = db[ID];
        await collection.find({}).forEach((item) => results.push([cut for charlimit]));
    }
    console.timeEnd("mongodb"); 
}

I've been trying out Javascript for my new backend, but I noticed something odd. When I use Python's PyMongo library, fetching all my data runs twice as fast (33.5s -> 16.44s) as when I use the official Javascript MongoDB module. The test setups are as follows:

mongo = MongoClient(URI) # initializing client

rn = time() # starting timer
for ID in LIST_OF_IDS:
    results = mongo["current_day_info"][ID].find() # fetches all the documents in the collection named <ID>
    results = [[result["time"].timestamp() / 10, result["buy_price"], result["sell_price"], result["buy_volume"], result["sell_volume"], result["week_buy_amount"], result["week_sell_amount"]] for result in results] # expands list of documents into array of their values
print("mongodb", time()-rn) # ending timer
const client = new MongoClient(URI); // initializing client

async function main() {
    await client.connect(); // connecting to client
    const database = client.db("current_day_info"); // goes to proper database

    console.time("mongodb"); // starting timer
    for (let ID of LIST_OF_IDS) {
        let results = [];
        const documents = database.collection(ID).find({}); // fetches all the documents in the collection named <ID>
        documents.forEach(item => results.push([new Date(item.time).getTime(), item.buy_price, item.sell_price, item.buy_volume, item.sell_volume, item.week_buy_amount, item.week_sell_amount])); // expands list of documents into array of their values
    }
    console.timeEnd("mongodb"); // ending timer
}

main();

My best guess is that PyMongo has some parts written in C, but I wouldn't think that results in such a drastic increase?

Versions:
Python 3.8.10, PyMongo 4.0.2, MongoDB version 5.0.6, MongoDB Node Driver version 4.4.1, NodeJS version 17.7.1


Copied from comments below:

Example document:

{
   "time":"ISODate(""2022-03-12T23:23:45.000Z"")",
   "_id":"ObjectId(""622d2b8c83d4c06e792895cb"")",
   "buy_volume":2079625,
   "sell_price":5.5,
   "sell_volume":10210328,
   "week_sell_amount":68184205,
   "week_buy_amount":10783950,
   "buy_price":7.4
}

Also, tried the same in MongoSH and it took significantly longer (7 minutes) I assume I messed something up there:

async function main() { 
    console.time("mongodb");
    for (let ID of LIST_OF_IDS) {
        let results = [];
        const collection = db[ID];
        await collection.find({}).forEach((item) => results.push([cut for charlimit]));
    }
    console.timeEnd("mongodb"); 
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文