序列化期间堆空间不足

发布于 2024-12-06 02:32:59 字数 469 浏览 2 评论 0原文

以下代码导致约 300 万行出现 OutOfMemmoryError: heap space

分配给 JVM 的内存为 4 GB,使用 64 位安装。

while (rs.next())
{    
    ArrayList<String> arrayList = new ArrayList<String>();
    for (int i = 1; i <= columnCount; i++)
    {
        arrayList.add(rs.getString(i));
    }

    objOS.writeObject(arrayList);
}

ArrayList 引用的内存可以在 while 循环的每次迭代中进行垃圾回收,并且在抛出 < code>OutOfMemoryError 因为堆空间。

那么为什么会出现异常呢?

The following code is causing a OutOfMemmoryError: heap space for some 3 million rows.

Memory allocated to JVM is 4 GB, using 64 bit installation.

while (rs.next())
{    
    ArrayList<String> arrayList = new ArrayList<String>();
    for (int i = 1; i <= columnCount; i++)
    {
        arrayList.add(rs.getString(i));
    }

    objOS.writeObject(arrayList);
}

The memory referenced by the ArrayList is eligible for garbage collection in each iteration of the while loop, and internally JVM calls garbage collection (System.gc()) before throwing an OutOfMemoryError because of heap space.

So why is the exception occurring?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

无妨# 2024-12-13 02:32:59

objOSObjectOutputStream 吗?

如果是这样,那么这就是您的问题:ObjectOutputStream 保留对曾经写入的每个对象的强引用,以避免两次写入同一个对象(它只会写一个引用说“我之前用 id x 编写的那个对象”)。

这意味着您实际上泄漏了所有 ArrayList 实例。

您可以通过调用 reset() 在您的 ObjectOutputStream 上。由于您似乎没有在 writeObject 调用之间使用该缓存,因此您可以在 writeObject() 之后直接调用 reset() > 打电话。

Is objOS an ObjectOutputStream?

If so, then that's your problem: An ObjectOutputStream keeps a strong reference to every object that was ever written to it in order to avoid writing the same object twice (it will simply write a reference saying "that object that I wrote before with id x").

This means that you're effectively leaking all ArrayList istances.

You can reset that "cache" by calling reset() on your ObjectOutputStream. Since you don't seem to be making use of that cache between writeObject calls anyway, you could call reset() directly after the writeObject() call.

妄断弥空 2024-12-13 02:32:59

我同意@Joachim 的观点。

下面的建议是一个神话

此外,建议(按照良好的编码约定)不要在循环内声明任何对象。相反,应在循环开始之前声明它,并使用相同的引用进行初始化。这将要求您的代码在每次迭代中使用相同的引用,并减轻内存释放线程(即垃圾收集)的负担。

真相
编辑了这个,因为我觉得可能有很多人(就像今天之前的我一样)仍然相信在循环内声明一个对象可能会损害内存管理;这是错误的。
为了演示这一点,我使用了 stackOverflow 上发布的相同代码< /a>.
以下是我的代码片段

package navsoft.advskill.test;

import java.util.ArrayList;

public class MemoryTest {

    /**
     * @param args
     */
    public static void main(String[] args) {
        /* Total number of processors or cores available to the JVM */
        System.out.println("Available processors (cores): "
                + Runtime.getRuntime().availableProcessors());
        /*
         * Total amount of free memory available to the JVM
         */
        long freeMemory = Runtime.getRuntime().freeMemory();
        System.out.println("Free memory (bytes): "
                + freeMemory);
        /*
         * This will return Long.MAX_VALUE if there is no preset limit
         */
        long maxMemory = Runtime.getRuntime().maxMemory();
        /*
         * Maximum amount of memory the JVM will attempt to use
         */
        System.out.println("Maximum memory (bytes): "
                + (maxMemory == Long.MAX_VALUE ? "no limit" : maxMemory));
        /*
         * Total memory currently in use by the JVM
         */
        System.out.println("Total memory (bytes): "
                + Runtime.getRuntime().totalMemory());
        final int LIMIT_COUNTER = 1000000;
        
        //System.out.println("Testing Only for print...");
        System.out.println("Testing for Collection inside Loop...");
        //System.out.println("Testing for Collection outside Loop...");
        //ArrayList<String> arr;
        for (int i = 0; i < LIMIT_COUNTER; ++i) {
            //arr = new ArrayList<String>();
            ArrayList<String> arr = new ArrayList<String>();
            System.out.println("" + i + ". Occupied(OldFree - currentFree): "+ (freeMemory - Runtime.getRuntime().freeMemory()));
        }
        System.out.println("Occupied At the End: "+ (freeMemory - Runtime.getRuntime().freeMemory()));
        System.out.println("End of Test");
    }

}

输出的结果清楚地表明,如果您在循环内部或外部声明对象,则占用/释放内存没有区别。因此建议声明的范围尽可能小。
我感谢 StackOverflow 上的所有专家(特别是 @Miserable Variable)在这方面对我的指导。

希望这也能消除您的疑虑。

I agree with @Joachim.

The below suggestion was a myth

In addition, it is recommended (in good coding convention) that do not declare any object inside the loop. Instead, declare it just before the loop start and use the same reference for initialization purpose. This will ask your code to use the same reference for each iterations and cause less burden on memory release thread (i.e. Garbage collection).

The Truth
I have edited this because I feel that there may be many people who (like me before today) still believe that declaring an object inside loop could harm the memory management; which is wrong.
To demonstrate this, I have used the same code posted on stackOverflow for this.
Following is my code snippet

package navsoft.advskill.test;

import java.util.ArrayList;

public class MemoryTest {

    /**
     * @param args
     */
    public static void main(String[] args) {
        /* Total number of processors or cores available to the JVM */
        System.out.println("Available processors (cores): "
                + Runtime.getRuntime().availableProcessors());
        /*
         * Total amount of free memory available to the JVM
         */
        long freeMemory = Runtime.getRuntime().freeMemory();
        System.out.println("Free memory (bytes): "
                + freeMemory);
        /*
         * This will return Long.MAX_VALUE if there is no preset limit
         */
        long maxMemory = Runtime.getRuntime().maxMemory();
        /*
         * Maximum amount of memory the JVM will attempt to use
         */
        System.out.println("Maximum memory (bytes): "
                + (maxMemory == Long.MAX_VALUE ? "no limit" : maxMemory));
        /*
         * Total memory currently in use by the JVM
         */
        System.out.println("Total memory (bytes): "
                + Runtime.getRuntime().totalMemory());
        final int LIMIT_COUNTER = 1000000;
        
        //System.out.println("Testing Only for print...");
        System.out.println("Testing for Collection inside Loop...");
        //System.out.println("Testing for Collection outside Loop...");
        //ArrayList<String> arr;
        for (int i = 0; i < LIMIT_COUNTER; ++i) {
            //arr = new ArrayList<String>();
            ArrayList<String> arr = new ArrayList<String>();
            System.out.println("" + i + ". Occupied(OldFree - currentFree): "+ (freeMemory - Runtime.getRuntime().freeMemory()));
        }
        System.out.println("Occupied At the End: "+ (freeMemory - Runtime.getRuntime().freeMemory()));
        System.out.println("End of Test");
    }

}

The result from the output is clearly shows that there is no difference in occupying/freeing the memory if you either declare the object inside or outside the loop. So it is recommended to have the declaration to as small scope as it can.
I pay my thanks to all the experts on StackOverflow (specially @Miserable Variable) for guiding me on this.

Hope this would clear your doubts too.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文