使用 C3P0 或 DBCP 的 Spring ORM 存在泄漏连接
在最新版本的 DBCP 和 C3P0 上,使用 Spring 的 Ibatis 支持,我遇到了两个泄漏连接的问题。
场景是有一条日志运行SQL,锁住了多个表。当用户触发命中锁定表的查询时,这会导致我的池中的连接达到最大。最后,管理员进入 MySQL 对长时间运行的 SQL 执行kill query
。
如果有足够的线程(在我的情况下大约 50 个或更多)正在等待将数据库线程检回到池中,那么我会在线程转储中看到类似以下内容:
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1315)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
- locked <0x00002aaacbb01118> (a com.mchange.v2.resourcepool.BasicResourcePool)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at
或者
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1104)
- locked <0x00002aab0f030620> (a org.apache.commons.pool.impl.GenericObjectPool$Latch)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at
这些线程永远等待。
如果池已达到最大,并且池中只有少数(大约 5 个)线程正在等待空闲连接,则不会发生这种情况。
我知道有配置可以解决这个问题(设置超时等),但我感兴趣的是为什么会发生这种情况?当有 50 个或更多线程等待连接并且我终止了长时间运行的 SQL 时,为什么活动线程没有返回到池中?
更新:我应该明确表示我正在使用 Spring 3.0.2 并且 伊巴蒂斯 2.3。我使用 SqlMapClientTemplate,它管理我的 对我来说有联系。此时,我开始认为这是 Ibatis 2.3 无法正确处理重负载。
On the latest versions of both DBCP and C3P0, using Spring's Ibatis support, I am running into an issue where both leak connections.
The scenario is that there is a log running SQL which locks multiple tables. This causes the connections in my pool to max out as users trigger queries which hit the locked tables. Finally, the administrator goes into MySQL and does a kill query <id>
on the long running SQL.
If there is enough threads (in my case around 50 or more) which are waiting around for a DB thread to be checked back in to the pool, then I see something like the following in a thread dump:
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1315)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
- locked <0x00002aaacbb01118> (a com.mchange.v2.resourcepool.BasicResourcePool)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at
or
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1104)
- locked <0x00002aab0f030620> (a org.apache.commons.pool.impl.GenericObjectPool$Latch)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at
and these threads wait around FOREVER.
This does not happen if the pool is maxed out, and only a few (around 5) threads are waiting for a free connection in the pool.
I know there is configuration that can fix this problem (setting timeout etc.), but I am interested in why this is happening in the first place? Why are the active threads not being returned to the pool, when there are 50 or more threads waiting for a connection and I kill the long running SQL?
Update: I should have made it clear that I am using Spring 3.0.2 and
Ibatis 2.3. I use the SqlMapClientTemplate, which manages my
connections for me. At this point, I am beginning to think it is
Ibatis 2.3 not handling heavy load correctly.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
只是添加到@BalusC的评论和@JoshDM的答案,当您在 Java 代码中对
Connection
调用close()
时,在幕后这实际上只会检查该连接放回到连接池中,而不是物理上关闭它。这就是为什么始终通过 Java 代码关闭连接非常重要,无论底层 JDBC 驱动程序是否正在池化连接。有关此问题的更多讨论,请参阅这篇文章。
Just to add to @BalusC's comment and @JoshDM's answer, when you call
close()
on aConnection
in your Java code, behind the scenes this will actually just check that connection back into your connection pool rather than physically close it. This is why it's important to always close connections through your Java code, regardless of whether the underlying JDBC driver is pooling them.More discussion about this on this post.
就像 @BalusC 所问的那样,您要关闭连接吗?它们应该在 Java
try-catch-finally
的finally
子句中关闭。将每个connection.close()
方法包装在其自己的try { con.close() } catch (Exceptionignore) {}
中,您也可以对
ResultSet 执行此操作
然后Statement
然后Connection
按这个顺序。在初始try
块之外将您的Connection
、Statement
和ResultSet
声明为null
并在您的try
块中实例化。如果您愿意,而不是在每个
finally
中包装 SQL 调用的所有上述代码,您可以选择使用 DbUtils Apache Commons DbUtils。并应用
closeQuietly
方法。finally
块将如下所示:Much like what @BalusC asks, are you closing your connections? They should be closed in the
finally
clause of your Javatry-catch-finally
. Wrap eachconnection.close()
method in its owntry { con.close() } catch (Exception ignore) {}
Optionally you should do so for
ResultSet
thenStatement
thenConnection
in that order. Declare yourConnection
,Statement
, andResultSet
outside your initialtry
block asnull
and instantiate in yourtry
block.If you're up for it, rather than all the above code in every
finally
that wraps a SQL call, you can opt to use the DbUtils utility class of Apache Commons DbUtils.and apply the
closeQuietly
methods. Thefinally
block will look like this instead:使用 c3po 时,我强烈建议尝试此功能。当您在事务之外执行数据库修改时,通常会出现连接泄漏。未提交写入的连接无法重用,并且会从池中丢失。如果您以相当长的超时启用此调试,您将能够看到可疑数据库操作的堆栈跟踪。
确保堆栈跟踪中看到的操作正确管理事务。还可以通过将 c3p0 记录器设置为调试级别来监视连接池的使用情况。
When using c3po, I strongly recommend trying this feature. Connection leaks usually appear when you perform database modifications outside transaction. Connection with uncommited writes cannot be reused and is lost from the pool. If you enable this debugging with reasonably long timeout, you will be able to see the stack traces with suspicious database operations.
Make sure the operations seen in the stack traces manage transactions correctly. Also monitor connection pool usage by setting c3p0 loggers to debug level.