如何加快查询响应时间?

发布于 2025-02-03 00:13:33 字数 2233 浏览 1 评论 0 原文

数据库中有一个集合,约有460万个文档。
(可以在屏幕截图中看到数据结构。)


When trying to pull at least one value from this collection, the response time to the request reaches more than 3 seconds:
(And this is on my local powerful machine.)

​ 处理代码请求和响应看起来像这样:

    // server.js
    const http = require('http');
    const { getProducts } = require('./controllers/currencyController');
    
    const server = http.createServer((req, res) => {
        if(req.url.match(/\/api\/currency.+/g) && req.method === 'GET') {
            getProducts(req, res);
        } else {
            res.writeHead(404, { 'Content-Type': 'application/json' })
            res.end(JSON.stringify({ message: 'Route Not Found' }))
        }
    })
    
    const PORT =  process.env.PORT || 5000
    server.listen(PORT, () => console.log(`Server running on port ${PORT}`))
    module.exports = server;


    //  currencyModel.js
    const {MongoClient} = require('mongodb');
    const client = new MongoClient('mongodb://127.0.0.1:27017/');
    
    const findAll = async (currencyPair) => {
        try{
            await client.connect();
            const testingData = client.db('Trading').collection(currencyPair);
            return testingData;
        }
        catch(e){
            console.log(e);
        }
    }

    module.exports = {
        findAll,
    }


// currencyController.js
const currencyData = require('../models/currencyModel')

async function getProducts(req, res) {
    try {
        const currency_data = await currencyData.findAll('EUR/USD');
        res.writeHead(200, { 'Content-Type': 'application/json' });
        res.end(JSON.stringify(await currency_data.find({id:1}).toArray()));
    } catch (error) {
        console.log(error)
    }
}

module.exports = {
    getProducts,
}

我怀疑这样一个长答案的原因可能与缺乏索引有关。 (存在的那些数字是自定义的。)

问题:
收到请求后,如何加快服务器的响应时间?
(最好不使用ORM和框架。)
这次理想情况下应该是什么?

There is a collection in the database, which has about 4.6 million documents.
(data structure can be seen in the screenshot.)

введите сюда описание изображения

When trying to pull at least one value from this collection, the response time to the request reaches more than 3 seconds:
(And this is on my local powerful machine.)

введите сюда описание изображения

Work with the code request and response looks like this:

    // server.js
    const http = require('http');
    const { getProducts } = require('./controllers/currencyController');
    
    const server = http.createServer((req, res) => {
        if(req.url.match(/\/api\/currency.+/g) && req.method === 'GET') {
            getProducts(req, res);
        } else {
            res.writeHead(404, { 'Content-Type': 'application/json' })
            res.end(JSON.stringify({ message: 'Route Not Found' }))
        }
    })
    
    const PORT =  process.env.PORT || 5000
    server.listen(PORT, () => console.log(`Server running on port ${PORT}`))
    module.exports = server;


    //  currencyModel.js
    const {MongoClient} = require('mongodb');
    const client = new MongoClient('mongodb://127.0.0.1:27017/');
    
    const findAll = async (currencyPair) => {
        try{
            await client.connect();
            const testingData = client.db('Trading').collection(currencyPair);
            return testingData;
        }
        catch(e){
            console.log(e);
        }
    }

    module.exports = {
        findAll,
    }


// currencyController.js
const currencyData = require('../models/currencyModel')

async function getProducts(req, res) {
    try {
        const currency_data = await currencyData.findAll('EUR/USD');
        res.writeHead(200, { 'Content-Type': 'application/json' });
        res.end(JSON.stringify(await currency_data.find({id:1}).toArray()));
    } catch (error) {
        console.log(error)
    }
}

module.exports = {
    getProducts,
}

I suspect that the reason for such a long answer may be somehow connected with the lack of indexes.
(those numbers that are present are custom created.)

Question:
How can I speed up the response time from the server after receiving the request?
(preferably without using ORM and frameworks.)
And what should ideally be this time?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

来日方长 2025-02-10 00:13:33

我正在根据评论获得的见解来编辑我的答案。

findall 方法有一些错误指示,因为它只是通过MongoDB驱动程序选择集合。问题是您正在触发表扫描,也就是说,MongoDB必须通过数据库中的每个行,以查看它是否匹配您的表达式ID:1。

您可以使用 findOne 而不是查找来缩短执行时间。一旦找到满足表达式的单个记录,它将结果返回。这通常还不够,好像匹配记录是数据库中的最后一个,性能将与 find 相同。

相反,您应该创建一个索引:

db.collection.createIndex({"id": 1})

在MongoDB控制台中运行此索引,或使用MongoDB指南针创建索引。该示例使用 ID 字段以升序顺序创建一个查询索引。

如果您想查询两个字段,例如VOL和CLOSS,VOL上升和关闭下降,则可以通过以下方式创建一个索引:

db.collection.createIndex({"vol": 1, "close":-1})

I'm editing my answer as per the insight gained though the comments.

The findAll method there is a bit of a misnomer, as it just is selecting a collection via the MongoDB driver. The problem is that you are triggering a table scan, that is, MongoDB has to go through every row in the database to see if it matches your expression id:1.

You can cut down the execution time by using findOne instead of find. Once a single record is found that satisfies the expression, it will return it as a result. This is usually not enough, as if the matching record is the last in the database, the performance will be the same as for find.

You should instead create an index:

db.collection.createIndex({"id": 1})

Run this in the MongoDB console, or use MongoDB Compass to create an index. The example creates an index for queries using the id field in ascending order.

If you wanted to query for two fields, say vol and close, vol ascending and close descending, you would create an index via:

db.collection.createIndex({"vol": 1, "close":-1})
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文