在java中压缩和解压大尺寸数据？

发布于 2024-11-28 05:23:10 字数 438 浏览 0 评论 0原文

我需要压缩/解压缩文件夹中包含的不同类型的文件，该文件夹的大小可能超过 10-11 GB。我使用了以下代码，但这需要很长时间来压缩数据。

BufferedReader in = new BufferedReader(new FileReader("D:/ziptest/expansion1.MPQ"));
BufferedOutputStream out = new BufferedOutputStream(
    new GZIPOutputStream(new FileOutputStream("test.gz")));

int c;
while ((c = in.read()) != -1)
  out.write(c);
in.close();
out.close();

请建议我一些java中的快速压缩和解压缩库，我也想将大文件分割成不同的部分，例如每个100MB的块。

原文

I need to compress/decompress different types of files that are contained in a Folder the size of that folder might be more than 10-11 GB.
I used following code but this is taking long time to compress the data.

BufferedReader in = new BufferedReader(new FileReader("D:/ziptest/expansion1.MPQ"));
BufferedOutputStream out = new BufferedOutputStream(
    new GZIPOutputStream(new FileOutputStream("test.gz")));

int c;
while ((c = in.read()) != -1)
  out.write(c);
in.close();
out.close();

Please suggest me some fast compressing and decompressing library in java, i also want to split the large file in different parts such as in a chunk of 100MB each.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

亚希 2024-12-05 05:23:10

读取器/写入器仅适用于文本，如果您尝试使用这些读取器/写入器读取二进制文件，则会损坏。

相反，我建议您使用 FileInputStream。复制数据最快的方法是使用您自己的缓冲区。

InputStream in = new FileInputStream("D:/ziptest/expansion1.MPQ");
OutputStream out = new GZIPOutputStream(
            new BufferedOutputStream(new FileOutputStream("test.gz")));

byte[] bytes = new byte[32*1024];
int len;
while((len = in.read(bytes)) > 0)
   out.write(bytes, 0, len);

in.close();
out.close();

由于您读取大块字节，因此不使用 BufferedInput/OuptuStream 会更有效，因为这会删除一份副本。 GZIPOutputStream 之后有一个 BufferedOutptuStream，因为您无法控制它生成的数据的大小。

顺便说一句：如果您仅使用 Java 阅读本文，则可以使用 DeflatorOutputStream，它稍快一些且更小，但仅受 Java AFAIK 支持。

Reader/Writer is only for Text and if you try to read binary with these is will get corrupted.

Instead I suggest you use FileInputStream. The fastest way to copy the data is to use your own buffer.

InputStream in = new FileInputStream("D:/ziptest/expansion1.MPQ");
OutputStream out = new GZIPOutputStream(
            new BufferedOutputStream(new FileOutputStream("test.gz")));

byte[] bytes = new byte[32*1024];
int len;
while((len = in.read(bytes)) > 0)
   out.write(bytes, 0, len);

in.close();
out.close();

Since you reading large chunks of bytes, it is more efficient not to BufferedInput/OuptuStream as this removes one copy. There is a BufferedOutptuStream after the GZIPOutputStream as you cannot control the size of data it produces.

BTW: If you are only reading this with Java, you can use DeflatorOutputStream, its slightly faster and smaller, but only supported by Java AFAIK.

回复收藏 0 原文

~没有更多了~