Sqlalchemy与Aurora无服务器V2 PostgreSQL-许多连接
我有一个从计算群集访问的AWS无服务器V2数据库设置（PostgreSQL）。该集群推出了大量的作业（＆GT; 1000），每个作业都独立地列出/从数据库中获取一些数据。根据需要，将无服务器群集设置为2至32个单位。
每个集群作业正在运行的代码使用SQLalchemy（ORM或Core）。我正在设置每个数据库连接，并使用一个空池和悲观的断开处理（即=“ nofollow noreferrer”> pool_pre_ping = true ）。从我对文档的阅读中，由于中间连接的闲置，应该处理断开连接。
还编写代码以访问数据库，获取结果，关闭连接（避免闲置连接），然后在处理后重新打开连接（5-30分钟）。这效果很好，因为一旦处理完成，新的连接就会交错，并且数据库已扩展。
我的日志正在显示标准，所有连接都是错误的：psycopg22.EperationalError：致命：剩余的连接插槽保留用于非复制超级用户和RDS_SUPERUSER连接，直到DB缩放可用的单元足够高。
问题：
- 我应该以不同的方式配置SQLalchemy连接吗？感觉就像是一个反模式，在等待数据库时缩放可用单元的数量，因为这种功能通常将其内置在Sqlalchemy中，以便抓住连接。
- 我是否应该使用 rds proxy 在数据库的前面？这似乎也像是一个反模式，在自动db的前面增加了代理。
PG版本为10。
I have an AWS Serverless V2 database setup (postgresql) that is being accessed from a compute cluster. The cluster launches a large number of jobs (>1000) and each job independently puts/pulls some data from the database. The Serverless cluster is setup to autoscale from 2 to 32 units as needed.
The code being run by each cluster job is using SQLAlchemy (either the ORM or the core). I am setting up each database connection with a null pool and pessimistic disconnect handling (i.e., pool_pre_ping=True). From my reading of the docs this should be handling disconnects due to being idle mid-connection.
Code is also written to access the DB, get the results, close the connection (to avoid idle connections), and then reopen the connection after processing (5-30 minutes). This is working well because once processing is completed, the new connections are staggered and the DB has scaled up.
My logs are showing the standard, all connections are taken error: psycopg2.OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser and rds_superuser connections until the DB scales the available units high enough.
Questions:
- Should I be configuring the SQLAlchemy connection differently? It feels like an anti-pattern to put in a custom retry to grab a connection while waiting for the DB to scale the number of available units as this type of capability seems to be built into SQLAlchemy usually.
- Should I be using an RDS Proxy in front of the database? This also seems like an anti-pattern, adding a proxy in front of an autoscaling DB.
PG version is 10.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论