Java 桌面应用程序的内存分析
我的应用程序加载大约的数据集。每次 85bm 至 100mb。该应用程序的内存限制设置为 512mb，理论上来说，这已经足够了。
然而，我发现，如果在应用程序的单次运行中，我打开和关闭数据集 5 次，总内存消耗会稳步增加，直到出现内存不足错误：
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6882 bguiz 20 0 679m 206m 19m S 30 13.7 0:30.22 java
6882 bguiz 20 0 679m 259m 19m S 9 17.2 0:55.53 java
6882 bguiz 20 0 679m 301m 19m S 9 20.0 1:20.04 java
6882 bguiz 20 0 679m 357m 19m S 33 23.7 1:44.74 java
6882 bguiz 20 0 679m 395m 19m S 80 26.2 2:10.31 java
内存从 ~14% 增长到〜26％。看起来像是内存泄漏。
发生的情况是，正在加载的顶级数据用于填充地图和列表等集合，然后使用更详细的数据创建这些顶级对象的子对象，然后它们依次创建子对象- 子对象。
当数据集关闭时，当前应用程序确实尝试通过取消填充各种对象集合来清除其轨迹，然后显式调用 System.gc();
无论如何，这是当我到达应用程序时的状态（在我之前已经准备好几年了），并且我被分配了这项任务。
我需要做的是找到一种方法来找到数据集卸载后哪些子对象和子子对象仍然互相引用，并纠正它们。
显然，这可以手动完成，但会非常非常繁琐，但我觉得通过内存分析来完成此操作会是一个更好的选择，这是我以前没有做过的事情。
我读过一些其他的 SO 问题，询问要使用哪种内存分析工具，我选择使用 Netbeans IDE 中内置的工具，因为它似乎有很好的评论，而且无论如何我都在 Netbeans 中工作。
以前是否有人承担过类似的 Java 内存分析任务，事后看来：
- 您会给我什么具体建议？
- 您发现哪些技术对解决这个问题有用？
- 您发现哪些资源对解决这个问题有用？
编辑： 该应用程序是标准桌面应用程序 - 不是 Web 应用程序。
编辑：已实现的解决方案
基本上对我有用的是将 Netbeans 的探查器与 JHAT 结合使用。
我发现 Netbeans IDE 中内置的探查器在特定处创建内存转储方面做得非常好>分析点，然后该工具能够按类进行过滤和排序，并深入了解每个实例的引用。这一切都非常好。
但是，它没有为我提供比较两个堆转储的方法。我问了一个后续问题，它看起来像 JHAT（作为一部分JDK）很好地完成了这项工作。
Thorbjørn Ravn Andersen、Dmitry 和 Jason Gritman：你们的意见非常有帮助，不幸的是我只能将 1 标记为正确答案，无论如何你们都从我这里得到了 +1。
My application loads a data set of approx. 85bm to 100mb each time. The application's memory limit is set to 512mb, and this is, theoretically, more than enough.
However, I found that if, in a single run of the application, I opened and closed the data set 5 times, the total memory consumption steadily increases, until I get an out-of-memory error:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6882 bguiz 20 0 679m 206m 19m S 30 13.7 0:30.22 java
6882 bguiz 20 0 679m 259m 19m S 9 17.2 0:55.53 java
6882 bguiz 20 0 679m 301m 19m S 9 20.0 1:20.04 java
6882 bguiz 20 0 679m 357m 19m S 33 23.7 1:44.74 java
6882 bguiz 20 0 679m 395m 19m S 80 26.2 2:10.31 java
Memory grew from ~14% to ~26%. It looks like a memory leak.
What's happening is that the top level data that is being loaded is used to populate collections such as maps and lists, and then the more detailed data is used to create sub-objects of these top-level objects, and then they in turn create sub-sub-objects.
When the data set is closed, currently the application does indeed make an attempt to clear its tracks by de-populating the various collections of objects, and then explicitly calling System.gc();
Anyhow, this is the state of the application when I got to it (several years in the making before me), and I have been assigned this task.
What I need to do, is to find a way to find which sub-objects and sub-sub-objects are still referencing each other after the data set is unloaded, and rectify them.
Obviously this can be done manually, but would be very very tedious, but I felt it would be a much better option to do this by memory profiling, something which I haven't done before.
I have read some other SO questions that asked about which memory profiling tools to use, and I have chosen to go with the one built into Netbeans IDE, since it seemed to have good reviews, and I am working in Netbeans anyway.
Has anyone undertaken a similar Java memory profiling task before, and with hindsight:
- What specific advice would you give me?
- What techniques did you find useful in tackling this problem?
- What resources did you find useful in tackling this problem?
Edit:
This application is a standard desktop application - not a web application.
Edit: Implemented solution
Basically what worked for me was to use Netbeans' profiler in conjunction with JHAT.
I found that the Profiler built into Netbeans IDE did a really good job of creating the memory dumps at particular profiling points, and then the tool was able to filter and sort by class and drill down the references for each instance. Which was all really good.
However, it didn't provide me with a means to compare two heap dumps. I asked a follow up question, and it looks like JHAT (comes as part of JDK) gets that job done quite well.
Thorbjørn Ravn Andersen, Dmitry and Jason Gritman: your input was really helpful, unfortunately I can only mark 1 as the correct answer, and all of you got +1 from me anyway.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
我在中写了另一个关于查找内存泄漏的技术的问题的答案https://stackoverflow.com/questions/1716597/java-memory-leak-detection-tools/1717260#1717260
如果您遵循我的建议，可以使用像 JProfiler 可以让您遍历对象的参考图并查看这些对象的深度大小。这可以帮助您找到仍然保留数据的任何对象。
我没有使用过 Netbeans，因此我无法告诉您它与我使用过的其他分析器相比如何。如果它看起来没有该功能，您可以轻松获得 JProfiler 的试用版，该试用版应该可以持续使用，直到您发现泄漏为止。
I wrote up an answer to another question about the techniques to go about finding memory leaks at https://stackoverflow.com/questions/1716597/java-memory-leak-detection-tools/1717260#1717260
If you follow my advice there, a tool like JProfiler can allow you to walk the reference graph of objects and view the deep size of those objects. This can help you find whatever object or objects are still holding onto the data.
I haven't worked with Netbeans, so I can't tell you how it stacks up against the other profilers I've used. If it doesn't look like it has that feature you can obtain a trial version of JProfiler easily, which should last you until you've found your leak.
使用 Java 6 JDK 中的 jvisualvm 连接到您的程序，看看您的内存去了哪里。
Attach to your program with jvisualvm from the Java 6 JDK and see where your memory goes.
NetBeans 探查器可能是免费探查器中最好的。它的工作很好，没有关于使用探查器本身的特殊提示。对于您的代码，请注意缓存某些内容（尤其是静态内容）的哈希图。它们常常是内存泄漏的根源。尝试使用 WeakHashMap 进行缓存（粗略地说，它不会创建对缓存值的强引用）。
NetBeans profiler is probably the best out of free ones. It does its job well, there are no special hints on working with the profiler itself. With respect to your code, pay attention to hashmaps where you cache something (especially static ones). Very often they are the sources of memory leaks. Try using WeakHashMap for caching (roughly, it doesn't create strong references to the cached values).
启动应用程序时是否更改内存分配？例如：
据我了解，当 JVM 无法足够快地分配内存时，也可能会发生内存不足错误。尽管它的上限为 512m（在上面的示例中），但如果 JVM 无法足够快地分配内存（超出上面的初始 128M），则可能会发生内存不足错误。如果问题确实存在，那么从较高的 -Xms 值开始可以缓解这种情况。还需要注意的是，Xms 和 Xmx 值只是建议，而不是硬性规定。
Are you altering the memory allocation when starting the application? For example:
It is my understanding that an out-of-memory error can also occur when the JVM is not able to allocate memory fast enough. Even though it has a ceiling of 512m (in the above example), if the JVM is not able to allocate memory fast enough (beyond the initial 128M above), an out-of-memory error could occur. Starting with a higher -Xms value could alleviate that if it were the problem. It is also important to note that the Xms and Xmx values are suggestions and not hard-and-fast rules.
查找用作缓存的静态集合（Map、Set、List）。
Look for static Collections (Map, Set, List) used as caches.
Eclipse Memory Analyzer 也是一个很好的分析堆转储的独立工具。
您有多种选择来创建堆转储（例如，在 OutOfMemoryExceptions 上）。
The Eclipse Memory Analyzer is also a good standalone tool to analyze heap dumps.
You have several options to create a dump of the heap (e.g. on OutOfMemoryExceptions).
您看到的行为不一定是内存泄漏。调用 System.gc() 只是向虚拟机暗示垃圾收集器应该运行（并非必须），只要堆有足够的可用空间，垃圾收集器通常不会运行。因此，看到进程大小增加并不能证明旧数据收集实际上不能被垃圾收集器回收。如果您不确定，我建议您对进程进行堆转储（具体方式取决于您使用的 Java VM，但其文档应该告诉您）并在堆转储上使用分析工具来查看是否有更多对象超出预期的实例保存在堆中以及引用它们的位置（这也可以解释内存泄漏的位置）。
作为第一次尝试，我可能会尝试使用新的 运行该程序G1 垃圾收集器，自 Java 1.6.0_14 起可用。在正常情况下，它可能更好地更早地到达可声明的实例，并且还具有能够将不需要的内存返回给操作系统的优点。其他Java垃圾收集器存在这样的问题：从操作系统分配的内存通常直到进程退出才返回。
The behaviour you are seeing is not necessarily a memory leak. Calling System.gc() is just a hint to the VM that the garbage collector should run (it does not have to) and as long as your heap has enough free space, the garbage collector usually does not run. So seeing the process size increasing is not a proof that the old data collections are not really claimable by the garbage collector. If you are not sure, I would recommend you to make a heap dump of the process (how depends on which Java VM you are using, but its documentation should tell you) and use an analysis tool on the heap dump to see if more object instances than expected are hold in the heap and from where they are referenced (this would also explain where the memory leak is).
As a first attempt, I would perhaps try to run the program with the new G1 garbage collector which is available since Java 1.6.0_14. Under normal circumstances it is probably better at reaching claimable instances earlier and also has the advantage of being able to return not required memory back to the operating system. Other Java garbage collectors have the problem that memory allocated from the OS is usually not returned until the process exits.