Tortoise-orm 在高并发下引发 KeyError

发布于 2025-01-12 20:48:29 字数 2505 浏览 2 评论 0原文

我在FastAPI中使用Tortoise-orm,Mysql连接池设置为{"minsize":8, "maxsize":16}

# models.py
class Device(models.Model):
    id = fields.IntField(pk=True)
    model = fields.CharField(max_length=32)
    version = fields.CharField(max_length=32)
        port = fields.IntField(default=0)
    
class Channel(models.Model):
    id = fields.IntField(pk=True)
    Device = fields.ForeignKeyField("model.Device", related_name="channels")
    frequency = fields.IntField(default=0, index=True)


#routers.py
@app.get("channels/{port}", response_model=ChannelSchema)
async def get_channels(port:int)
        return await ChannelSchema.from_queryset(Channel.filter(Device__port=port))

我使用JMeter测试了这个API:

第一次是每秒100个接口请求:50%的请求是500错误,

查看服务日志:

  File "/root/TuffyProjects/simulator/venv/lib/python3.8/site-packages/tortoise/models.py", line 742, in _init_from_db

KeyError: 'model'

或者

File "/root/TuffyProjects/simulator/venv/lib/python3.8/site-packages/tortoise/models.py", line 742, in _init_from_db

KeyError: 'frequency'

第二次是每秒100个接口请求:20%的请求是500错误。

第三次是每秒100个接口请求:5%的请求是500个错误。

第四次是每秒100个接口请求:5%的请求是500个错误。

当我增加并发请求数时,错误数又回到了 50%。

我将重新启动服务并重新测试它。

第一次是每秒100个接口请求:50%的请求是500错误,看服务器日志发现KeyError提示不是model或者Frequency而是两个其他领域。

我把连接池设置改为minsize":100, "maxsize":100},启动项目时Mysql日志提示:

2022-03-09T07:09:24.855819Z 116 [Note] Aborted connection 116 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.855712Z 78 [Note] Aborted connection 78 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.855140Z 124 [Note] Aborted connection 124 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.856009Z 123 [Note] Aborted connection 123 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:34.903747Z 108 [Note] Aborted connection 108 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:34.903765Z 104 [Note] Aborted connection 104 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)

但是服务运行正常,并发请求数减少,

错误是什么?这个BUG的原因有没有更好的解决办法呢?

I use Tortoise-orm in FastAPI, Mysql connection pool settings is {"minsize":8, "maxsize":16}

# models.py
class Device(models.Model):
    id = fields.IntField(pk=True)
    model = fields.CharField(max_length=32)
    version = fields.CharField(max_length=32)
        port = fields.IntField(default=0)
    
class Channel(models.Model):
    id = fields.IntField(pk=True)
    Device = fields.ForeignKeyField("model.Device", related_name="channels")
    frequency = fields.IntField(default=0, index=True)


#routers.py
@app.get("channels/{port}", response_model=ChannelSchema)
async def get_channels(port:int)
        return await ChannelSchema.from_queryset(Channel.filter(Device__port=port))

I tested this API using JMeter:

The first time is 100 interface requests per second:50% of requests is 500 error,

Viewing service logs:

  File "/root/TuffyProjects/simulator/venv/lib/python3.8/site-packages/tortoise/models.py", line 742, in _init_from_db

KeyError: 'model'

or

File "/root/TuffyProjects/simulator/venv/lib/python3.8/site-packages/tortoise/models.py", line 742, in _init_from_db

KeyError: 'frequency'

The second time is 100 interface requests per second:20% of requests is 500 error.

The third time is 100 interface requests per second:5% of requests is 500 error.

The fourth time is 100 interface requests per second:5% of requests is 500 error.

When I increased the number of concurrent requests, the number of errors went back up to 50%.

I will restart the service and re-test it.

The first time is 100 interface requests per second:50% of requests is 500 error,A look at the server logs shows that the KeyError prompt is not modelor frequency but two other fields.

I changed the connection pool settings to minsize":100, "maxsize":100}, Mysql log prompt when starting project:

2022-03-09T07:09:24.855819Z 116 [Note] Aborted connection 116 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.855712Z 78 [Note] Aborted connection 78 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.855140Z 124 [Note] Aborted connection 124 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:24.856009Z 123 [Note] Aborted connection 123 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:34.903747Z 108 [Note] Aborted connection 108 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)
2022-03-09T07:09:34.903765Z 104 [Note] Aborted connection 104 to db: 'Device' user: 'root' host: '127.0.0.1' (Got an error reading communication packets)

But the service runs normally and the number of concurrent requests errors is reduced.

What is the cause of this BUG? Is there a better solution?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文