Java内存不足异常
我正在 Tomcat 中运行 Java Web 应用程序。该应用程序使用 Quartz 框架定期安排 cron 作业。这个 cron 作业涉及解析一个 4+ MB 的 xml 文件，这是我使用 JDOM API 完成的。 xml 文件包含大约 3600 个要解析的节点，因此数据要在数据库中更新，我正在按顺序执行此操作。
解析几乎一半的文件后，我的应用程序抛出内存不足异常。相同的堆栈跟踪是：
Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.traceVMObjectAlloc(ProfilerRuntimeMemory.java:170)
at java.lang.Throwable.getStackTraceElement(Native Method)
at java.lang.Throwable.getOurStackTrace(Throwable.java:590)
at java.lang.Throwable.getStackTrace(Throwable.java:582)
at org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:155)
at org.apache.juli.logging.DirectJDKLog.error(DirectJDKLog.java:135)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1603)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)
at java.lang.Thread.run(Thread.java:619)
Exception in thread "*** JFluid Monitor thread ***" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2760)
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.Vector.ensureCapacityHelper(Vector.java:226)
at java.util.Vector.add(Vector.java:728)
at org.netbeans.lib.profiler.server.Monitors$SurvGenAndThreadsMonitor.updateSurvGenData(Monitors.java:230)
at org.netbeans.lib.profiler.server.Monitors$SurvGenAndThreadsMonitor.run(Monitors.java:169)
Nov 30, 2009 2:22:05 PM org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
SEVERE: Exception invoking periodic operation:
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:232)
at java.lang.StringCoding.encode(StringCoding.java:272)
at java.lang.String.getBytes(String.java:946)
at java.io.UnixFileSystem.getLastModifiedTime(Native Method)
at java.io.File.lastModified(File.java:826)
at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1175)
at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1269)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:296)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:118)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1337)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1601)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)
at java.lang.Thread.run(Thread.java:619)
ERROR [JobRunShell]: Job updateVendorData.quoteUpdate threw an unhandled Exception:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.apache.commons.dbcp.PoolingConnection$PStmtKey.hashCode(PoolingConnection.java:296)
at java.util.HashMap.get(HashMap.java:300)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.decrementActiveCount(GenericKeyedObjectPool.java:1085)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:882)
at org.apache.commons.dbcp.PoolablePreparedStatement.close(PoolablePreparedStatement.java:80)
at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:168)
at com.netcore.smsapps.stock.db.CompanyDaoImpl.updateCompanyQuote(CompanyDaoImpl.java:173)
at com.netcore.smsapps.stock.vendor.MyirisVendor.readScripQuotes(MyirisVendor.java:159)
at com.netcore.smsapps.stock.update.StockUpdateData.execute(StockUpdateData.java:38)
at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
DEBUG [ExceptionHelper]: Detected JDK support for nested exceptions.
ERROR [ErrorLogger]: Job (updateVendorData.quoteUpdate threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Java heap space]
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.apache.commons.dbcp.PoolingConnection$PStmtKey.hashCode(PoolingConnection.java:296)
at java.util.HashMap.get(HashMap.java:300)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.decrementActiveCount(GenericKeyedObjectPool.java:1085)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:882)
at org.apache.commons.dbcp.PoolablePreparedStatement.close(PoolablePreparedStatement.java:80)
at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:168)
at com.netcore.smsapps.stock.db.CompanyDaoImpl.updateCompanyQuote(CompanyDaoImpl.java:173)
at com.netcore.smsapps.stock.vendor.MyirisVendor.readScripQuotes(MyirisVendor.java:159)
at com.netcore.smsapps.stock.update.StockUpdateData.execute(StockUpdateData.java:38)
at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
这甚至导致我的 tomcat 崩溃。您能帮我诊断问题吗？我什至在 Netbeans 中启用了同样的分析，但似乎即使这样也崩溃了。我保留了分配给 Tomcat 的默认内存。是否发生内存泄漏。 我的数据库是postgres，JDK是1.6.0_15。
谢谢， 阿米特
I am running a Java Web Application in Tomcat. The application uses Quartz framework to schedule the cron job at regular intervals. This cron job involves parsing a 4+ MB xml file, which I am doing using JDOM API. The xml file contains around 3600 nodes to be parsed and consequently data to be updated in DB which I am doing it sequentially.
After parsing almost half of the file, my application throws a Out of Memory Exception. The stack trace of the same is :
Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.traceVMObjectAlloc(ProfilerRuntimeMemory.java:170)
at java.lang.Throwable.getStackTraceElement(Native Method)
at java.lang.Throwable.getOurStackTrace(Throwable.java:590)
at java.lang.Throwable.getStackTrace(Throwable.java:582)
at org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:155)
at org.apache.juli.logging.DirectJDKLog.error(DirectJDKLog.java:135)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1603)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)
at java.lang.Thread.run(Thread.java:619)
Exception in thread "*** JFluid Monitor thread ***" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2760)
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.Vector.ensureCapacityHelper(Vector.java:226)
at java.util.Vector.add(Vector.java:728)
at org.netbeans.lib.profiler.server.Monitors$SurvGenAndThreadsMonitor.updateSurvGenData(Monitors.java:230)
at org.netbeans.lib.profiler.server.Monitors$SurvGenAndThreadsMonitor.run(Monitors.java:169)
Nov 30, 2009 2:22:05 PM org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
SEVERE: Exception invoking periodic operation:
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:232)
at java.lang.StringCoding.encode(StringCoding.java:272)
at java.lang.String.getBytes(String.java:946)
at java.io.UnixFileSystem.getLastModifiedTime(Native Method)
at java.io.File.lastModified(File.java:826)
at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1175)
at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1269)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:296)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:118)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1337)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1601)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1610)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1590)
at java.lang.Thread.run(Thread.java:619)
ERROR [JobRunShell]: Job updateVendorData.quoteUpdate threw an unhandled Exception:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.apache.commons.dbcp.PoolingConnection$PStmtKey.hashCode(PoolingConnection.java:296)
at java.util.HashMap.get(HashMap.java:300)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.decrementActiveCount(GenericKeyedObjectPool.java:1085)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:882)
at org.apache.commons.dbcp.PoolablePreparedStatement.close(PoolablePreparedStatement.java:80)
at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:168)
at com.netcore.smsapps.stock.db.CompanyDaoImpl.updateCompanyQuote(CompanyDaoImpl.java:173)
at com.netcore.smsapps.stock.vendor.MyirisVendor.readScripQuotes(MyirisVendor.java:159)
at com.netcore.smsapps.stock.update.StockUpdateData.execute(StockUpdateData.java:38)
at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
DEBUG [ExceptionHelper]: Detected JDK support for nested exceptions.
ERROR [ErrorLogger]: Job (updateVendorData.quoteUpdate threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Java heap space]
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3210)
at java.lang.String.<init>(String.java:216)
at java.lang.StringBuffer.toString(StringBuffer.java:585)
at org.apache.commons.dbcp.PoolingConnection$PStmtKey.hashCode(PoolingConnection.java:296)
at java.util.HashMap.get(HashMap.java:300)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.decrementActiveCount(GenericKeyedObjectPool.java:1085)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.returnObject(GenericKeyedObjectPool.java:882)
at org.apache.commons.dbcp.PoolablePreparedStatement.close(PoolablePreparedStatement.java:80)
at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:168)
at com.netcore.smsapps.stock.db.CompanyDaoImpl.updateCompanyQuote(CompanyDaoImpl.java:173)
at com.netcore.smsapps.stock.vendor.MyirisVendor.readScripQuotes(MyirisVendor.java:159)
at com.netcore.smsapps.stock.update.StockUpdateData.execute(StockUpdateData.java:38)
at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
This causes even my tomcat to crash. Can you please help me in diagnosing the problem. I even have enabled profiling in the Netbeans for the same but it seems that even that crashed. I have kept the default memory allocated to Tomcat. Is there any memory leak taking place.
My DB is postgres and JDK is 1.6.0_15.
Thanks,
Amit
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（9）
尝试增加 JVM 的内存分配。应该有帮助。
修复 Eclipse：您可以在 Eclipse 首选项中进行配置，如下所示
Try increasing the ram allocation for your JVM. It should help.
Fix for eclipse: You can configure this in eclipse preference as follows
每次使用 DOM 解析 XML 文件时，您都会将整个文件加载到内存中，并且 DOM 基础结构将使用大约相同的大小来处理它，因此它消耗的内存大约是文件大小的两倍。
您需要使用 SAX，一个基于事件的解析器。虽然第一次可能很难理解，但它的内存效率非常高，因为它只是将当前解析节点保留在内存中。
似乎Java有一些SAX实现，比如StAX，我希望它有帮助。
Everytime you use a DOM to parse a XML file, you'll load entire file into memory and DOM infrastructure will use about same size to handle it, so it'll consume about twice memory than your file size.
You'll need to use SAX, an event based parser. While this can be hard to understand it first time, it's a very memory effective, as it just keeps in memory current parsing node.
Seems Java have some SAX implementations, like StAX, I hope it helps.
解析 XML 是一项相当昂贵的任务。一般的 DOM 解析器需要的内存空间至少是 XML 文档大小的五倍。您也应该考虑到这一事实。为了确保其他地方没有内存泄漏导致 XML 解析器内存不足，您确实需要运行探查器。给它更多的内存，将可用内存加倍并对其进行分析。当您确定原因并修复泄漏后，您可以退回到“默认”内存并重新测试。或者，如果确实没有任何泄漏的方法，那么只需给它比默认值多一点的内存，这样它就适合了。
您还可以考虑使用内存效率更高的 XML 解析器，例如 VTD-XML (主页位于此处，此处为基准）。
Parsing XML is an fairly expensive task. The average DOM parser would already need at least five times of the memory space as the XML document big is. You should take this fact into account as well. To ensure that there is no memory leak somewhere else which caused the memory shortage for the XML parser, you really need to run a profiler. Give it all more memory, double the available memory and profile it. When you've nailed the cause down and fixed the leak, then you can just fall back to the "default" memory and retest. Or if there is really no means of any leak, then just give it all a bit more memory than default so that it all suits.
You can also consider to use a more memory efficient XML parser instead, for example VTD-XML (homepage here, benchmarks here).
尝试增加 JVM 的内存分配。应该有帮助。
修复 Eclipse
您可以在 Eclipse 的首选项中进行如下配置：
Windows ->首选项（在 Mac 上为：Eclipse -> 首选项） Java ->安装的 JRE
选择 JRE，然后单击
-Xms256m -Xmx512m -XX:MaxPermSize=512m -XX:PermSize=128m中的默认 VM 参数字段类型上的“编辑”（或您的内存首选项，对于 1 GB 的内存，它是 1024）。单击完成或确定。Try increasing the ram allocation for your JVM. It should help.
Fix for Eclipse
You can configure this in Eclipse's preferences as follows:
Windows -> Preferences (on Mac it's: Eclipse -> Preferences) Java -> Installed JREs
Select the JRE and click on Edit on the default VM arguments field type in
-Xms256m -Xmx512m -XX:MaxPermSize=512m -XX:PermSize=128m(or your memory preference, for 1 gb of ram it's 1024). Click on finish or OK.您必须为 tomcat JVM 的 PermGenSpace 分配更多空间。
这可以通过 JVM 参数来完成：
-XX:MaxPermSize=128m默认情况下，PermGen 空间是 64M（并且它包含所有已编译的类，所以如果你有很多 jar（类））你的类路径，你确实可以填充这个空间）。
顺便说一句，您可以使用 JVisualVM 监视 PermGen 空间的大小，甚至可以使用以下命令检查其内容YourKit Java 分析器
You have to allocate more space to the PermGenSpace of the tomcat JVM.
This can be done with the JVM argument :
-XX:MaxPermSize=128mBy default, the PermGen space is 64M (and it contains all compiled classes, so if you have a lot of jar (classes) in your classpath, you may indeed fill this space).
On a side note, you can monitor the size of the PermGen space with JVisualVM and you can even inspect its content with YourKit Java Profiler
您是否尝试将最大堆大小设置得更大，看看问题是否仍然出现？甚至可能根本就没有泄漏。可能只是默认堆大小（我认为 Windows 上为 64m）不足以满足此特定进程的需要。
我发现我几乎总是需要为运行 Tomcat 的任何应用程序提供比默认值更多的堆和永久代空间，否则我会遇到内存不足的问题。如果您需要调整内存设置的帮助，请查看 这个问题。
Have you tried setting the max heap size bigger to see if the problem still occurs then? There may not even be a leak at all. It might just be that the default heap size (64m on Windows I think) is insufficient for this particular process.
I find that I almost always need to give any application I'm running Tomcat more heap and perm gen space than the defaults or I'll run into out of memory problems. If you need help adjusting the memory settings take a look at this question.
您确定某处没有递归数组副本被错误地留在那里吗？也许在不同的线程中？
Are you sure there are no recursive array copy somewhere, left there by mistake? Perhaps in different threads?
我会附和关于文件和 DOM 占用大量内存的观点。当我看到这个的时候我也想知道：
那个复制是做什么的？我想知道您的代码中是否还存在其他问题。
如果您已经完成了这一步，则表明您已成功读取文件和 DOM，并开始写入数据库。文件内存应该已经被回收。
我建议使用 VisualGC 查看内存，这样你就可以看到发生了什么。
I'll second that point about the file and the DOM taking up a great deal of memory. I also wonder when I see this:
What's that copying doing? I wonder if there's something else bad going on in your code.
If you've gotten this far, it suggests that you've read the file and the DOM successfully and you're starting to write to the database. The file memory should already be reclaimed.
I'd suggest looking at memory using VisualGC so you can see what's going on.
您可以使用以下命令运行应用程序：-XX:+HeapDumpOnOutOfMemoryError。这将导致 JVM 在内存不足时生成堆转储。您可以使用诸如 MAT 或 JHAT 之类的东西来查看所持有的对象。我建议在生成的堆转储上使用 Eclipse 内存分析器工具 (MAT)，因为它使用起来相当简单： http ://www.eclipse.org/mat/
当然，您需要了解哪些对象可能徘徊在周围，才能发挥作用。 DOM 对象？来自先前加载的 xml 文档的资源？数据库连接？ MAT 将允许您从您怀疑应该被垃圾收集的某个对象追溯到根对象的引用。
You could run your application with: -XX:+HeapDumpOnOutOfMemoryError. This will cause the JVM to produce a heap dump when it runs out of memory. You can the use something like: MAT or JHAT to see what objects are being held on to. I suggest using the eclipse memory analyzer tool (MAT) on the generated heap dump as it is fairly straightforward to use: http://www.eclipse.org/mat/
Of course you will need to have some idea as to what objects may be hanging around in order for this to be useful. DOM objects? Resources from previous loads of xml documents? Database connections? MAT will allow you to trace the references back to a root object from some object that you suspect should have been garbage collected.