将大文件（>300MB）上传到时出现 OutOfMemoryErrors
使用 Commons FileUpload 1.2.1 的 servlet。这看起来很奇怪，因为
使用 DiskFileItem 的全部目的是防止（可能
大）文件驻留在内存中。我使用的是默认尺寸
阈值为 10KB，因此这就是应该加载到
堆，对吗？这是部分堆栈跟踪：
java.lang.OutOfMemoryError
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:177)
at org.apache.commons.fileupload.disk.DiskFileItem.get(DiskFileItem.java:334)
at org.springframework.web.multipart.commons.CommonsMultipartFile.getBytes(CommonsMultipartFile.java:114)
为什么会发生这种情况？我缺少一些配置吗？ 除了增加堆大小之外，还有什么技巧/技巧可以避免这种情况吗？
我真的不应该增加我的堆，因为理论上应该从这个操作加载到内存中的最多是 10KB 多一点。另外，我的最大堆 (-Xmx) 已经设置为 1GB，这应该足够了。
I am getting OutOfMemoryErrors when uploading large (>300MB) files to
a servlet utilizing Commons FileUpload 1.2.1. It seems odd, because
the entire point of using DiskFileItem is to prevent the (possibly
large) file from residing in memory. I am using the default size
threshold of 10KB, so that's all that should ever be loaded into the
heap, right? Here is the partial stack trace:
java.lang.OutOfMemoryError
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:177)
at org.apache.commons.fileupload.disk.DiskFileItem.get(DiskFileItem.java:334)
at org.springframework.web.multipart.commons.CommonsMultipartFile.getBytes(CommonsMultipartFile.java:114)
Why is this happening? Is there some configuration I'm missing? Any tips/tricks to avoid this situation besides increasing my heap size?
I really shouldn't have to increase my heap, because in theory the most that should be loaded into memory from this operation is a little over 10KB. Plus, my heap max (-Xmx) is already set for 1GB which should be plenty.
发布评论
评论（2）
在处理文件上传（尤其是大文件上传）时，您应该将这些文件作为流处理，将其放入中等大小的内存缓冲区中，然后直接复制到输出文件中。错误的做法是把整个事情吸入内存然后再写出来。
commons-upload 上的文档 在中间下方提到了如何“处理文件”上传”。如果您记得以合理大小的块（例如 1 MB）从输入流复制到输出流，那么应该没有问题。
When dealing with file uploads, especially big ones, you should process those files as streams which you slurp into a medium-size in-memory buffer and copy directly into your output file. The wrong way to do it is to inhale the whole thing into memory before writing it out.
The doc on commons-upload mentions, just below the middle, how to "Process a file upload". If you remember to copy from the inputstream to the outputstream in reasonably sized chunks (say, 1 MB), you should have no problem.
虽然 Carl Smotricz 的答案在一般情况下可能更好，但您得到的例外是此处报告的 JVM 错误：
https://bugs.java.com/bugdatabase/view_bug?bug_id=6478546
While Carl Smotricz's answer is probably better in the general case, the exception you get is a JVM bug that is reported here:
https://bugs.java.com/bugdatabase/view_bug?bug_id=6478546