Readline 太慢 - 有更快的吗?

发布于 2024-12-09 10:23:16 字数 505 浏览 1 评论 0原文

我正在使用 BufferedReader 和 InputStreamReader 从流中读取内容,以创建从读取器创建的一个长字符串。它的行数超过 100,000 行,然后抛出 500 错误(服务器上的调用失败)。我不确定问题是什么,有没有比这种方法更快的方法?当线路数达到数千时它可以工作,但我正在处理大型数据集。

BufferedReader in = new BufferedReader(new InputStreamReader(newConnect.getInputStream()));
String inputLine;               
String xmlObject = "";
StringBuffer str = new StringBuffer();

while ((inputLine = in.readLine()) != null) {
    str.append(inputLine);
    str.toString();
}       
in.close();

提前致谢

I am reading in from a stream using a BufferedReader and InputStreamReader to create one long string that gets created from the readers. It gets up to over 100,000 lines and then throws a 500 error (call failed on the server). I am not sure what is the problem, is there anything faster than this method? It works when the lines are in the thousands but i am working with large data sets.

BufferedReader in = new BufferedReader(new InputStreamReader(newConnect.getInputStream()));
String inputLine;               
String xmlObject = "";
StringBuffer str = new StringBuffer();

while ((inputLine = in.readLine()) != null) {
    str.append(inputLine);
    str.toString();
}       
in.close();

Thanks in advance

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

苦妄 2024-12-16 10:23:16

创建一个由读者创建的长字符串。

您是否有机会这样做来创建您的“长字符串”?

String string;
while(...) 
 string+=whateverComesFromTheSocket;

如果是,则将其更改为

StringBuilder str = new StringBuilder(); //Edit:Just changed StringBuffer to StringBuilder
while(...)
 str.append(whateverComesFromTheSocket);
String string = str.toString(); 

String 对象是不可变的,并且当您执行 str+="something" 时,会重新分配内存,并将 str+"something" 复制到新分配的区域。这是一项成本高昂的操作,运行 51,000 次是一件极其糟糕的事情。

StringBuffer和StringBuilder是String的可变兄弟,StringBuilder是非并发的,比StringBuffer效率更高。

to create one long string that gets created from the readers.

Are you by any chance doing this to create your "long string"?

String string;
while(...) 
 string+=whateverComesFromTheSocket;

If yes, then change it to

StringBuilder str = new StringBuilder(); //Edit:Just changed StringBuffer to StringBuilder
while(...)
 str.append(whateverComesFromTheSocket);
String string = str.toString(); 

String objects are immutable and when you do str+="something", memory is reallocated and str+"something" is copied to that newly allocated area. This is a costly operation and running it 51,000 times is an extremely bad thing to do.

StringBuffer and StringBuilder are String's mutable brothers and StringBuilder, being non-concurrent is more efficient than StringBuffer.

戏舞 2024-12-16 10:23:16

readline() 的读取速度约为 90 MB/s,这是您对数据读取的操作,速度很慢。 BTW readline 删除了换行符,因此您使用的这种方法是有缺陷的,因为它会将每一行变成一行。

我建议您不要重新发明轮子 FileUtils.readLineToString()
这将有效地将文件读取为字符串而不丢弃换行符。

readline() can read at about 90 MB/s, its what you are doing with the data read which is slow. BTW readline removes newlines so this approach you are using is flawed as it will turn everying into one line.

Rather than re-inventing the wheel I would suggest you try FileUtils.readLineToString()
This will read a file as a STring without discarding newlines, efficiently.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文