为什么 Java VM 在“打开文件过多”后无法恢复？错误？
在某些众所周知的情况下，我们的应用程序将打开太多套接字（数据库连接）并达到操作系统允许的最大打开文件数。我们明白这一点；我们正在解决这个问题并提高限制。
我们无法解释的是，为什么即使连接数量减少并且我们完全在限制范围内，我们的应用程序的某些部分也无法恢复。
在本例中，它是在 Tomcat 下运行的应用程序。
发生这种情况时，我们首先开始看到“打开文件过多”错误：
SEVERE: Socket accept failed
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:310)
at java.lang.Thread.run(Thread.java:619)
最终，我们开始在尝试打开 HTTP 连接的应用程序线程内看到 NoClassDefFoundError：
java.lang.NoClassDefFoundError: org/apache/commons/httpclient/protocol/ControllerThreadSocketFactory
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:128)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1349)
[...]
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 8 more
当错误连接消失时，服务器再次开始接受连接，一切似乎都正常，但我们留下后一个错误不断地被发送到 stderr。
尽管应用程序通常将卸载的类记录到标准输出，但在“打开文件过多”错误之前、期间或之后我没有看到任何此类日志。
我最初的理论是，Hotspot JVM 在遇到“打开文件过多”时会卸载看似未使用的类，但如果是这样，它不会记录这一事实。
编辑： 正如 Stephen C 在下面指出的，如果它正在卸载类，并且在第一次重新加载时遇到错误，这可以解释为什么它永远不会恢复。我认为这是一个很好的工作理论。 Sun 文档中有记录吗？为什么它不会像通常卸载类的方式那样记录该类正在被卸载？
平台详情：
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) 64-Bit Server VM (build 14.0-b16, mixed mode)
Apache Tomcat Version 6.0.18
In certain well-understood circumstances, our application will open too many sockets (database connections) and reach the maximum open files that the OS allows. We understand this; we are fixing the issue and also bumping up the limit.
What we can't explain is why parts of our application don't recover even after the number of connections abates and we're well within the limit.
In this case, it's an application running under Tomcat.
When this happens, we first start seeing "Too many open files" errors:
SEVERE: Socket accept failed
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:310)
at java.lang.Thread.run(Thread.java:619)
Eventually, we start seeing NoClassDefFoundErrors inside an application thread that's trying to open HTTP connections:
java.lang.NoClassDefFoundError: org/apache/commons/httpclient/protocol/ControllerThreadSocketFactory
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:128)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1349)
[...]
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 8 more
When the errant connections go away, the server starts accepting connections again, and everything seems ok, but we're left with the latter error constantly being spewed to stderr.
Although the application typically logs unloaded classes to stdout, I don't see any such logs just before, during or after the "Too many open files" errors.
My initial theory was that the Hotspot JVM would unload seemingly unused classes when it encounters "Too many open files," but if so, it doesn't log the fact.
Edit: As Stephen C indicates below, if it is unloading the class, and encounters an error the first time it reloads, that could explain why it never recovers. I think that's a good working theory. Is it documented in the Sun docs? Why would it not log that the class is being unloaded the way unloading a class usually is?
Platform details:
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) 64-Bit Server VM (build 14.0-b16, mixed mode)
Apache Tomcat Version 6.0.18
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我认为您重复出现 ClassNotFoundExceptions 的原因是，由于 Socket 泄漏问题，第一次尝试的 ControllerThreadSocketFactory 类初始化失败。您的代码现在重复执行重新触发类的类初始化的操作，并且它们正在报告原始问题。
如果类初始化第一次失败，那就是这样。 JVM 不会再尝试这样做。
I think that the reason you are getting repeated ClassNotFoundExceptions is that the first attempted class initialization of ControllerThreadSocketFactory failed due to the Socket leakage problem. Your code is now repeatedly doing things that is retriggering class initialization for the class, and they are reporting the original problem.
If a class initialization fails first time, that's it. The JVM will not try to do it again.
使用 Weblogic 8.1 / JRockIt R27.2 和一堆尝试加载资源包的 Web 应用程序面临同样的问题，然后由于打开文件数量的限制而失败。停止和启动应用程序（即卸载和加载类加载器）会使事情再次正常运行。
Facing the same issue using Weblogic 8.1 / JRockIt R27.2 and a bunch of webapps that tries to load resourcebundles and then fails due to the limit on the number of open files. Stopping and starting the application (i.e. unloading and loading classloaders) make things works again.