java中如何在没有内存问题的情况下下载大文件

发布于 2024-11-30 08:48:43 字数 804 浏览 0 评论 0原文

当我尝试从服务器下载 260MB 的大文件时,出现以下错误:java.lang.OutOfMemoryError: Java heap space. 我确信我的堆大小小于 252MB。有什么方法可以在不增加堆大小的情况下下载大文件吗?

如何下载大文件而不出现此问题?我的代码如下:

String path= "C:/temp.zip";   
response.addHeader("Content-Disposition", "attachment; filename=\"test.zip\""); 
byte[] buf = new byte[1024];   
try {   

             File file = new File(path);   
             long length = file.length();   
             BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));   
             ServletOutputStream out = response.getOutputStream();   

             while ((in != null) && ((length = in.read(buf)) != -1)) {   
             out.write(buf, 0, (int) length);   
             }   
             in.close();   
             out.close();

When I am trying to download a large file which is of 260MB from server, I get this error: java.lang.OutOfMemoryError: Java heap space. I am sure my heap size is less than 252MB. Is there any way I can download large files without increasing heap size?

How I can download large files without getting this issue? My code is given below:

String path= "C:/temp.zip";   
response.addHeader("Content-Disposition", "attachment; filename=\"test.zip\""); 
byte[] buf = new byte[1024];   
try {   

             File file = new File(path);   
             long length = file.length();   
             BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));   
             ServletOutputStream out = response.getOutputStream();   

             while ((in != null) && ((length = in.read(buf)) != -1)) {   
             out.write(buf, 0, (int) length);   
             }   
             in.close();   
             out.close();

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

明媚如初 2024-12-07 08:48:43

我可以看到您可能在两个地方增加内存使用量:

  1. 在读取输入文件的缓冲区中。
  2. 在写入输出流的缓冲区中(HTTPOutputStream?)

对于#1,我建议通过 FileInputStream 直接从文件中读取,而不使用 BufferedInputStream。请先尝试此操作,看看是否可以解决您的问题。即:

FileInputStream in = new FileInputStream(file);   

而不是:

BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));   

如果#1 不能解决问题,您可以尝试在写入大量数据后定期刷新输出流(如有必要,减少块大小):

即:

try
{
    FileInputStream fileInputStream  = new FileInputStream(file);
    byte[] buf=new byte[8192];
    int bytesread = 0, bytesBuffered = 0;
    while( (bytesread = fileInputStream.read( buf )) > -1 ) {
        out.write( buf, 0, bytesread );
        bytesBuffered += bytesread;
        if (bytesBuffered > 1024 * 1024) { //flush after 1MB
            bytesBuffered = 0;
            out.flush();
        }
    }
}
finally {
    if (out != null) {
        out.flush();
    }
}

There are 2 places where I can see you could potentially be building up memory usage:

  1. In the buffer reading your input file.
  2. In the buffer writing to your output stream (HTTPOutputStream?)

For #1 I would suggest reading directly from the file via FileInputStream without the BufferedInputStream. Try this first and see if it resolves your issue. ie:

FileInputStream in = new FileInputStream(file);   

instead of:

BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));   

If #1 does not resolve the issue, you could try periodically flushing the output stream after so much data is written (decrease chunk size if necessary):

ie:

try
{
    FileInputStream fileInputStream  = new FileInputStream(file);
    byte[] buf=new byte[8192];
    int bytesread = 0, bytesBuffered = 0;
    while( (bytesread = fileInputStream.read( buf )) > -1 ) {
        out.write( buf, 0, bytesread );
        bytesBuffered += bytesread;
        if (bytesBuffered > 1024 * 1024) { //flush after 1MB
            bytesBuffered = 0;
            out.flush();
        }
    }
}
finally {
    if (out != null) {
        out.flush();
    }
}
不再让梦枯萎 2024-12-07 08:48:43

不幸的是,您没有提到 out 是什么类型。如果您有内存问题,我猜是 ByteArrayOutpoutStream 。因此,将其替换为 FileOutputStream 并将要下载的字节直接写入文件。

顺便说一句,不要使用逐字节读取的 read() 方法。请改用read(byte[] arr)。这要快得多。

Unfortunately you have not mentioned what type out is. If you have memory issues I guess it is ByteArrayOutpoutStream. So, replace it by FileOutputStream and write the byte you are downloading directly to file.

BTW, do not use read() method that reads byte-by-byte. Use read(byte[] arr) instead. This is much faster.

寒江雪… 2024-12-07 08:48:43

首先,您可以从 while 语句中删除 (in != null) ,这是不必要的。其次,尝试删除 BufferedInputStream 并执行以下操作:

FileInputStream in = new FileInputStream(file);

First you can remove the (in != null) from your while statement, it's unnecessary. Second, try removing the BufferedInputStream and just do:

FileInputStream in = new FileInputStream(file);
時窥 2024-12-07 08:48:43

您显示的代码没有任何问题(就内存使用而言)。 servlet 容器配置为缓冲整个响应(查看 web.xml 配置),或者内存在其他地方泄漏。

There's nothing wrong (in regard to memory usage) with the code you're show. Either the servlet container is configured to buffer the entire response (look at the web.xml configuration), or the memory is being leaked elsewhere.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文