尝试在内存中创建大型 zip 文件时出现内存分配错误

发布于 2025-01-16 12:19:22 字数 790 浏览 1 评论 0原文

一旦遇到 for 循环，内存分配错误似乎就会发生，并且仅适用于非常大的 storeList。只有当storesList.size()大约为200万时，我才能遇到内存分配错误。我粘贴了 for 循环的片段，该循环应该根据循环中的每个对象创建一个新行。

该函数接受 ZipOutputStream 和对象列表。

 public ZipOutPutStream setStream(ZipoutputStream zos, List<Stores> storesList){
     zos.setLevel(1);
     newRow = new StringBuilder();
      for (Stores s: storesList) {
        //clearing the stringbuilder each iteration
        newRow.setLength(0);
        newRow.append(s.getId());
        newRow.append(",")
            .append(s.getTitle()).append(s.getDescription());
        newRow.append("\n");
      }
      byte[] data = result.toString().getBytes();
      zos.write(data, 0, data.length);
      zos.closeEntry();
      zos.close();
      return zos;
}

我应该改变什么才能处理非常大的列表？

原文

The memory allocation error specifically seems to occur once it hits the for loop and only for very large storesLists. I only able to hit the memory allocation error if storesList.size() is about 2 million. I've pasted a snipped of the for loop which is supposed to create a new row based off each object in the loop.

The function takes in a ZipOutputStream and a list of objects.

 public ZipOutPutStream setStream(ZipoutputStream zos, List<Stores> storesList){
     zos.setLevel(1);
     newRow = new StringBuilder();
      for (Stores s: storesList) {
        //clearing the stringbuilder each iteration
        newRow.setLength(0);
        newRow.append(s.getId());
        newRow.append(",")
            .append(s.getTitle()).append(s.getDescription());
        newRow.append("\n");
      }
      byte[] data = result.toString().getBytes();
      zos.write(data, 0, data.length);
      zos.closeEntry();
      zos.close();
      return zos;
}

What should I be changing so that I am able to process very large lists?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

做个少女永远怀春 2025-01-23 12:19:22

当您可以直接传输相同的数据时，不要构建内存结构 - 因为巨大的结构可能会在 toString() 或 getBytes()OutOfMemoryError代码>调用。

如果您正在处理平台行分隔符，您可以像这样写出行：

PrintWriter w = new PrintWriter(zos/*,false, orCharSetHere*/);
for (Stores s : storesList) {
    w.println(s.getId() + "," + s.getTitle() + s.getDescription());
}
w.flush();

但是如果您想要所有平台的 \n 行分隔符，则可以使用以下方式发送行：

OutputStreamWriter w = new OutputStreamWriter(zos/*, orCharSetHere*/);
for (Stores s : storesList) {
    w.write(s.getId() + "," + s.getTitle() + s.getDescription()+"\n");
}
w.flush();

必须刷新写入器以确保任何缓冲在 zip 条目或流关闭之前，数据会写入 ZipOutputStream。

Don't build a memory structure when you can stream the same data directly - as huge structures may hit OutOfMemoryError on the toString() or getBytes() calls.

If you are dealing with platform line separators you can write out rows like this:

PrintWriter w = new PrintWriter(zos/*,false, orCharSetHere*/);
for (Stores s : storesList) {
    w.println(s.getId() + "," + s.getTitle() + s.getDescription());
}
w.flush();

But if you want \n line separators for all platforms the rows can be sent using:

OutputStreamWriter w = new OutputStreamWriter(zos/*, orCharSetHere*/);
for (Stores s : storesList) {
    w.write(s.getId() + "," + s.getTitle() + s.getDescription()+"\n");
}
w.flush();

The writers must be flushed to ensure any buffered data is written to the ZipOutputStream before the zip entry or stream is closed.

回复收藏 0 原文

~没有更多了~