我的方法是出于某种原因即使在完成后使用记忆

发布于 2025-02-11 10:59:45 字数 1418 浏览 4 评论 0原文

我正在编程一个神经网络,其中我将权重数据保存在一个长txt文件中,因此要检索我使用的数据'|'将数字分开,然后在返回数组之前添加到数组中。

使用此方法,我的RAM用法转到1500 MB,并且直到整个程序结束,但我的RAM使用情况才会消失,我的RAM使用将转移到700

我试图关闭所有内容,也许认为我的一个对象是一个线程,但这也没有用。我的代码中是否有任何可能导致此RAM用法的内容,

这是代码:

public static int[] findDoub(String fileName) {
    ArrayList<Integer> innt = new ArrayList<Integer>();
    try {
        File file = new File(fileName);
        FileInputStream strem = new FileInputStream(file);
        BufferedInputStream buff = new BufferedInputStream(strem);
        
        int index = buff.read();
        StringBuilder doubus = new StringBuilder("");
        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus.append(a);
                } else {
                    innt.add(Integer.valueOf(doubus.toString()));
                    doubus = new StringBuilder("");
                }
            }
            index = buff.read();
        }
        buff.close();
        strem.close();
        buff = null;
        strem = null;
        
    } catch (IOException e) {
        e.printStackTrace();
    }
    Runtime.getRuntime().gc();
    int[] innnt = new int[innt.size()];
    for (int i = 0; i < innnt.length; i++) {
        innnt[i] = innt.get(i);
    }
    return innnt;
}

I was programming a neural network in which I save the data of the weights in one long txt file and so to retrieve that data I used '|' to separate numbers and then added into an array before returning it.

Using this method my ram usage goes to 1500 MB and doesn't go away until the whole program ends but without it my ram usage goes to 700.

I tried to close everything, maybe thinking one of my objects is a thread, but that didn't work either. Is there anything in my code that could be causing this ram usage

Here's the code :

public static int[] findDoub(String fileName) {
    ArrayList<Integer> innt = new ArrayList<Integer>();
    try {
        File file = new File(fileName);
        FileInputStream strem = new FileInputStream(file);
        BufferedInputStream buff = new BufferedInputStream(strem);
        
        int index = buff.read();
        StringBuilder doubus = new StringBuilder("");
        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus.append(a);
                } else {
                    innt.add(Integer.valueOf(doubus.toString()));
                    doubus = new StringBuilder("");
                }
            }
            index = buff.read();
        }
        buff.close();
        strem.close();
        buff = null;
        strem = null;
        
    } catch (IOException e) {
        e.printStackTrace();
    }
    Runtime.getRuntime().gc();
    int[] innnt = new int[innt.size()];
    for (int i = 0; i < innnt.length; i++) {
        innnt[i] = innt.get(i);
    }
    return innnt;
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

橙味迷妹 2025-02-18 10:59:45

在几个地方使用了过多的内存,但是方法中的所有内容除了返回的int []是可收集的垃圾,因此您不应有任何疑虑。

但是,如果您以许多值阅读 - 说100,000或更多,那么以下建议将减少所需的内存足迹。

主张ArrayList在使用前避免重新分配时:

int initialSize = 100000;
ArrayList<Integer> innt = new ArrayList<>(initialSize);

避免每次解析Integer避免使用StringBuffer。由于int具有最大长度,因此您可以替换为final char [] dourus

final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
int dIndex = 0;
// ... Then append with:
    if (a != '|') {
        doubus[dIndex++] = a;
    } else {
        innt.add(Integer.valueOf(new String(doubus, 0, dIndex)));
        dIndex=0;
    }

使用integer.valueofarraylist&lt; integer&gt; 表示您将自动键入许多int值作为整数,仅在末尾提取为int。将arraylist交换为int []并使用integer.parseint,因此结果始终是原始类型int是指避免了许多内存转换:

int [] innt = new int[initialSize];
int iIndex = 0;
// Replace add with:
    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));

将其放在一起您应该具有相同的输出,希望记忆力减少:

public static int[] findDoub(File file) throws IOException {
    int initialSize = 100000; // Use suitable value
    int [] innt = new int[initialSize];
    int iIndex = 0;

    try (BufferedInputStream buff = new BufferedInputStream(new FileInputStream(file))) {
        int index = buff.read();

        final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
        int dIndex = 0;

        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus[dIndex++] = a;
                } else {
                    // Grow int[] if needed:
                    if (iIndex == innt.length) {
                        innt = Arrays.copyOf(innt, innt.length + initialSize);
                    }
                    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));
                    dIndex=0;
                }
            }
            index = buff.read();
        }
    }
    // Return exact size int[] of current length:
    return Arrays.copyOf(innt, iIndex);
}

There are several places where excessive memory is used but everything within the method except the returned int[] is garbage collectable so you should not have any concerns.

However If you are reading in many values - say 100,000 or more then the suggestions below will reduce the memory footprint needed.

Presize ArrayList before use avoids re-allocations when it grows:

int initialSize = 100000;
ArrayList<Integer> innt = new ArrayList<>(initialSize);

Avoid StringBuffer per parsed Integer. As int has maximum length you can replace with final char[] doubus:

final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
int dIndex = 0;
// ... Then append with:
    if (a != '|') {
        doubus[dIndex++] = a;
    } else {
        innt.add(Integer.valueOf(new String(doubus, 0, dIndex)));
        dIndex=0;
    }

Using Integer.valueOf with ArrayList<Integer> means you are auto-boxing many int values as Integer, only to extract as int at the end. Swap out the ArrayList for int[] and use Integer.parseInt so the result is always primitive type int means many memory conversions are avoided:

int [] innt = new int[initialSize];
int iIndex = 0;
// Replace add with:
    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));

Putting this together you should have same output and hopefully less memory churn:

public static int[] findDoub(File file) throws IOException {
    int initialSize = 100000; // Use suitable value
    int [] innt = new int[initialSize];
    int iIndex = 0;

    try (BufferedInputStream buff = new BufferedInputStream(new FileInputStream(file))) {
        int index = buff.read();

        final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
        int dIndex = 0;

        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus[dIndex++] = a;
                } else {
                    // Grow int[] if needed:
                    if (iIndex == innt.length) {
                        innt = Arrays.copyOf(innt, innt.length + initialSize);
                    }
                    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));
                    dIndex=0;
                }
            }
            index = buff.read();
        }
    }
    // Return exact size int[] of current length:
    return Arrays.copyOf(innt, iIndex);
}
单身情人 2025-02-18 10:59:45

您将所有文件读取到INNT,直到功能结束为止。然后您将其复制到Innnt。两者

都引起RAM使用。

RAM使用也取决于您的文件大小。

您不需要复制Innt,只需返回ArrayList,它就会改善RAM使用情况。

您可以尝试将文件分开。

you read all the file to innt until the function end. and you copy it to innnt. the

both causing ram usage.

ram usage also depends on you file size.

you dont need copy innt, just return arraylist, it will improve ram usage.

and you can try split the file.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文