当前位置： 文江博客话题详情

Android：读取txt文件很慢并且将数据插入trie时内存泄漏（错误的hashmap使用）？

发布于 2024-12-08 08:21:24 字数 4690 浏览 0 评论 0原文

我正在尝试在我的 android 应用程序中读取 816 KB txt 文件（70 000 行），该文件包含在 res/raw 文件夹中。这是相关的代码片段：

            InputStream raw = context.getResources().openRawResource(R.raw.alpha);
            InputStreamReader inputreader = new InputStreamReader(raw);
            BufferedReader r = new BufferedReader(inputreader);
            String line;

            int j = 0;
            while ((line = r.readLine()) != null && j < 90000){
                theTrie.insert(line);
                j++;
            }

“上下文”是通过参数给出的：Main.readDatabase(getApplication());

这个版本崩溃并显示以下日志：

10-04 21:14:26.096: INFO/dalvikvm(1327):   | group="main" sCount=0 dsCount=0 obj=0x4001f1a8 self=0xce48
10-04 21:14:26.096: INFO/dalvikvm(1327):   | sysTid=1327 nice=0 sched=0/0 cgrp=default handle=-1345006528
10-04 21:14:26.096: INFO/dalvikvm(1327):   | schedstat=( 3112456663 436561269 431 )
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.nativeFillInStackTrace(Native Method)
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.fillInStackTrace(Throwable.java:135)
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.<init>(Throwable.java:74)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.Error.<init>(Error.java:38)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:36)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:34)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.util.HashMap.makeTable(HashMap.java:-1)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.util.HashMap.<init>(HashMap.java:162)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.TrieNode.<init>(TrieNode.java:21)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.Trie.insert(Trie.java:27)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.Main.readDatabase(Main.java:356)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.SoftKeyboard.onCreate(SoftKeyboard.java:89)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.handleCreateService(ActivityThread.java:1920)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.access$2500(ActivityThread.java:117)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:982)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.os.Handler.dispatchMessage(Handler.java:99)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.os.Looper.loop(Looper.java:123)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.main(ActivityThread.java:3647)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.reflect.Method.invokeNative(Native Method)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.reflect.Method.invoke(Method.java:507)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at dalvik.system.NativeStart.main(Native Method)
10-04 21:14:26.196: WARN/dalvikvm(1327): Exception thrown (Ljava/lang/OutOfMemoryError;) while throwing internal exception (Ljava/lang/OutOfMemoryError;)
10-04 21:14:26.326: INFO/dalvikvm-heap(1327): Clamp target GC heap from 25.881MB to 24.000MB
10-04 21:14:26.326: DEBUG/dalvikvm(1327): GC_CONCURRENT freed <1K, 13% free 20224K/22983K, external 1625K/2137K, paused 2ms+3ms
10-04 21:14:26.446: INFO/dalvikvm-heap(1327): Clamp target GC heap from 25.856MB to 24.000MB
10-04 21:14:26.446: DEBUG/dalvikvm(1327): GC_FOR_MALLOC freed 25K, 13% free 20198K/22983K, external 1625K/2137K, paused 123ms
10-04 21:14:26.446: DEBUG/AndroidRuntime(1327): Shutting down VM

如果我不在模拟器中运行此代码（由于文件读取而略有改变），但在普通的 java 应用程序中它运行良好，没有任何问题，但在 Android 上似乎存在内存泄漏？有人知道可能是什么问题吗？ 即使我删除了“theTrie.insert(line);”行因此，while 循环（几乎）为空，应用程序需要 2.1 秒来“读取”800 kb。这不可能是正常的吧？我的错误在哪里？

提前致谢！

更新： 我用 DDMS 转储了一个堆文件并用 MAT 对其进行了分析。它说：由“dalvik.system.PathClassLoader @ 0x4051d5e8”加载的类[有问题]占用 23.951.808 (91,76%) 字节。内存累积在“”加载的“java.util.HashMap$HashMapEntry[]”的一个实例中。 直方图列出了 java.util.HashMap$HashMapEntry[]、java.util.HashMap 和 java.util.HashMap$HashMapEntry 作为浅堆最大的三个对象。

这是我关于 hashmap 使用的代码（在定义 trie 节点的类中）：

public Map<Character, TrieNode> children;

public TrieNode(char letter) {
    this.letter = letter;
    children = new HashMap<Character, TrieNode>(26);
}

是什么导致了内存泄漏？

原文

I'm trying to read a 816 KB txt file (70 000 lines) in my android app which is contained in the res/raw folder. This is the relevant code snippet:

            InputStream raw = context.getResources().openRawResource(R.raw.alpha);
            InputStreamReader inputreader = new InputStreamReader(raw);
            BufferedReader r = new BufferedReader(inputreader);
            String line;

            int j = 0;
            while ((line = r.readLine()) != null && j < 90000){
                theTrie.insert(line);
                j++;
            }

The "context" is given via a parameter: Main.readDatabase(getApplication());

This version crashes with the following log:

10-04 21:14:26.096: INFO/dalvikvm(1327):   | group="main" sCount=0 dsCount=0 obj=0x4001f1a8 self=0xce48
10-04 21:14:26.096: INFO/dalvikvm(1327):   | sysTid=1327 nice=0 sched=0/0 cgrp=default handle=-1345006528
10-04 21:14:26.096: INFO/dalvikvm(1327):   | schedstat=( 3112456663 436561269 431 )
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.nativeFillInStackTrace(Native Method)
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.fillInStackTrace(Throwable.java:135)
10-04 21:14:26.096: INFO/dalvikvm(1327):   at java.lang.Throwable.<init>(Throwable.java:74)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.Error.<init>(Error.java:38)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:36)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:34)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.util.HashMap.makeTable(HashMap.java:-1)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.util.HashMap.<init>(HashMap.java:162)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.TrieNode.<init>(TrieNode.java:21)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.Trie.insert(Trie.java:27)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.Main.readDatabase(Main.java:356)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at snappy.keyboard.SoftKeyboard.onCreate(SoftKeyboard.java:89)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.handleCreateService(ActivityThread.java:1920)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.access$2500(ActivityThread.java:117)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:982)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.os.Handler.dispatchMessage(Handler.java:99)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.os.Looper.loop(Looper.java:123)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at android.app.ActivityThread.main(ActivityThread.java:3647)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.reflect.Method.invokeNative(Native Method)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at java.lang.reflect.Method.invoke(Method.java:507)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
10-04 21:14:26.106: INFO/dalvikvm(1327):   at dalvik.system.NativeStart.main(Native Method)
10-04 21:14:26.196: WARN/dalvikvm(1327): Exception thrown (Ljava/lang/OutOfMemoryError;) while throwing internal exception (Ljava/lang/OutOfMemoryError;)
10-04 21:14:26.326: INFO/dalvikvm-heap(1327): Clamp target GC heap from 25.881MB to 24.000MB
10-04 21:14:26.326: DEBUG/dalvikvm(1327): GC_CONCURRENT freed <1K, 13% free 20224K/22983K, external 1625K/2137K, paused 2ms+3ms
10-04 21:14:26.446: INFO/dalvikvm-heap(1327): Clamp target GC heap from 25.856MB to 24.000MB
10-04 21:14:26.446: DEBUG/dalvikvm(1327): GC_FOR_MALLOC freed 25K, 13% free 20198K/22983K, external 1625K/2137K, paused 123ms
10-04 21:14:26.446: DEBUG/AndroidRuntime(1327): Shutting down VM

If I don't run this code (slightly changed due to file reading) in the emulator but in a normal java application it runs fine without any problems but on Android there seems to be a memory leak? Does somebody have any idea what could be the problem?
Even if I delete the line "theTrie.insert(line);" so that the while loop is (nearly) empty the app needs 2,1 seconds for "readlining" the 800 kb. This can't be normal, right? Where is my mistake?

Thanks in advance!

Update:
I dumped a heap file with DDMS and analyzed it with MAT. It says: The class [in question], loaded by "dalvik.system.PathClassLoader @ 0x4051d5e8", occupies 23.951.808 (91,76%) bytes. The memory is accumulated in one instance of "java.util.HashMap$HashMapEntry[]" loaded by "".
The histogram lists java.util.HashMap$HashMapEntry[], java.util.HashMap and java.util.HashMap$HashMapEntry as the three objects with the largest shallow heap.

Here is my code regarding the hashmap use (in the class thats defining the node of the trie):

public Map<Character, TrieNode> children;

public TrieNode(char letter) {
    this.letter = letter;
    children = new HashMap<Character, TrieNode>(26);
}

Whats causing the memory leak?

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（1）

绿阴红影里的.如风往事 2024-12-15 08:21:24

有时模拟器并不像我们中的一些人想要的那么准确。有时，某些在终端中运行良好的进程在模拟器中运行缓慢。

使用 Android 手机尝试一下，然后别忘了告诉我们您的体验。

-------------------------------------------------------- 修改

这个：

        BufferedReader r = new BufferedReader(new InputStreamReader(context.getResources().openRawResource(R.raw.alpha)));
        String line = null;

        int j = 0;
        while ((line = r.readLine()) != null && j < 90000){
            theTrie.insert(line);
            j++;
        }

然后看看你调用这个函数多少次，也许这就是问题所在。

但是，思考并查看 trie 地图，我也可以告诉您对您正在阅读的内容进行投资，也许行太长了。

-------------------------------------------------------- 修改 2< /strong>

我认为你能做的最好的事情就是改变观点并尝试使用 SQLite 数据库来获得如此多的信息并且没有内存问题。

Sometimes the emulator is not so much accurate than some of we would want . Sometimes some process which runs nicely in a terminal , runs slow in emulator .

Try it with an android's phone and then don't forget to tell us with your experience.

-------------------------------------------- MODIFY

Try this:

        BufferedReader r = new BufferedReader(new InputStreamReader(context.getResources().openRawResource(R.raw.alpha)));
        String line = null;

        int j = 0;
        while ((line = r.readLine()) != null && j < 90000){
            theTrie.insert(line);
            j++;
        }

And then see how many times do you call this function , maybe that's the problem.

But , thinking about and seeing trie map , i can tell you too make an invest about what are you reading ,maybe the lines are too long .

-------------------------------------------- MODIFY 2

I think the best you can do is change the point of view and try to use a SQLite db to have so much info and without problems of memory.

回复 原文
~没有更多了~

关于作者

短叹

暂无简介

0 文章
0 评论
24 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

小瓶盖

文章 0 评论 0

wxsp_Ukbq8xGR

文章 0 评论 0

1638627670

文章 0 评论 0

仅一夜美梦

文章 0 评论 0

夜访吸血鬼

文章 0 评论 0

近卫軍团

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文