Apache Commons Net FTP 正在上传损坏的文件
我正在尝试使用 Apache Commons Net 进行 FTP 文件传输。
问题是文件间歇性到达服务器时已损坏。我所说的“损坏”是指 WinRAR 告诉我 ZIP 文件有一个“存档意外结束”。有时文件完全是空的。我注意到对于较大的文件(100kb+)来说这种情况更常见,但是对于小文件(20kb)也会发生这种情况。
我知道上传的源 zip 文件是有效的,并且只有 243kb。
我没有从代码中得到任何错误/异常。
下面是正在执行的代码:
int CON_TIMEOUT = (int) TimeUnit.SECONDS.toMillis(20); // fail if can't connect within 20 seconds
int LIVE_TIMEOUT = (int) TimeUnit.MINUTES.toMillis(5); // allow up to 5 minutes for data transfers
FTPClient client = new FTPClient();
client.setConnectTimeout(CON_TIMEOUT);
client.setDataTimeout(LIVE_TIMEOUT);
client.connect(host);
client.setSoTimeout(LIVE_TIMEOUT);
client.login(user, pass);
client.changeWorkingDirectory(dir);
log("client ready");
File file = new File(filePath);
String name = new Date().getTime() + "-" + file.getName();
InputStream fis = null;
try
{
fis = new FileInputStream(file);
if (!client.storeFile(name, fis))
throw new RuntimeException("store failed");
log("store " + name + " complete");
}
finally
{
IOUtils.closeQuietly(fis);
try
{
client.logout();
log("logout");
}
catch (Throwable e)
{
log("logout failed", e);
}
try
{
client.disconnect();
log("disconnect");
}
catch (Throwable e)
{
log("disconnect failed", e);
}
}
和一些日志:
2010-08-10 21:32:38 client ready
2010-08-10 21:32:49 store 1281439958234-file.zip complete
2010-08-10 21:32:49 logout
2010-08-10 21:32:49 disconnect
2010-08-10 21:32:50 client ready
2010-08-10 21:33:00 store 1281439970968-file.zip complete
2010-08-10 21:33:00 logout
2010-08-10 21:33:00 disconnect
2010-08-10 21:33:02 client ready
2010-08-10 21:33:11 store 1281439982234-file.zip complete
2010-08-10 21:33:11 logout
2010-08-10 21:33:11 disconnect
2010-08-10 21:33:15 client ready
2010-08-10 21:33:25 store 1281439995890-file.zip complete
2010-08-10 21:33:26 logout
2010-08-10 21:33:26 disconnect
2010-08-10 21:33:27 client ready
2010-08-10 21:33:36 store 1281440007531-file.zip complete
2010-08-10 21:33:36 logout
2010-08-10 21:33:36 disconnect
2010-08-10 21:33:37 client ready
2010-08-10 21:33:48 store 1281440017843-file.zip complete
2010-08-10 21:33:48 logout
2010-08-10 21:33:48 disconnect
2010-08-10 21:33:49 client ready
2010-08-10 21:33:59 store 1281440029781-file.zip complete
2010-08-10 21:33:59 logout
2010-08-10 21:33:59 disconnect
2010-08-10 21:34:00 client ready
2010-08-10 21:34:09 store 1281440040812-file.zip complete
2010-08-10 21:34:09 logout
2010-08-10 21:34:09 disconnect
2010-08-10 21:34:10 client ready
2010-08-10 21:34:23 store 1281440050859-file.zip complete
2010-08-10 21:34:24 logout
2010-08-10 21:34:24 disconnect
2010-08-10 21:34:25 client ready
2010-08-10 21:34:35 store 1281440065421-file.zip complete
2010-08-10 21:34:35 logout
2010-08-10 21:34:35 disconnect
请注意,所有这些都在 15 秒内完成,并且服务器上的所有生成文件都已损坏。
我也测试过,没有设置任何超时,问题仍然出现。
I'm trying to use Apache Commons Net for FTP file transfers.
Problem is files are intermittently arriving at the server corrupt. By 'corrupt' I mean that WinRAR tells me a ZIP file has an 'Unexpected end of archive'. Sometimes the files are completely empty. I have noticed that this happens more for larger files (100kb+), however does happen for small files too (20kb).
I know for a fact that the source zip file being uploaded is valid, and is only 243kb.
I do not get any errors/exceptions from the code.
Here's the code being executed:
int CON_TIMEOUT = (int) TimeUnit.SECONDS.toMillis(20); // fail if can't connect within 20 seconds
int LIVE_TIMEOUT = (int) TimeUnit.MINUTES.toMillis(5); // allow up to 5 minutes for data transfers
FTPClient client = new FTPClient();
client.setConnectTimeout(CON_TIMEOUT);
client.setDataTimeout(LIVE_TIMEOUT);
client.connect(host);
client.setSoTimeout(LIVE_TIMEOUT);
client.login(user, pass);
client.changeWorkingDirectory(dir);
log("client ready");
File file = new File(filePath);
String name = new Date().getTime() + "-" + file.getName();
InputStream fis = null;
try
{
fis = new FileInputStream(file);
if (!client.storeFile(name, fis))
throw new RuntimeException("store failed");
log("store " + name + " complete");
}
finally
{
IOUtils.closeQuietly(fis);
try
{
client.logout();
log("logout");
}
catch (Throwable e)
{
log("logout failed", e);
}
try
{
client.disconnect();
log("disconnect");
}
catch (Throwable e)
{
log("disconnect failed", e);
}
}
and some logs:
2010-08-10 21:32:38 client ready
2010-08-10 21:32:49 store 1281439958234-file.zip complete
2010-08-10 21:32:49 logout
2010-08-10 21:32:49 disconnect
2010-08-10 21:32:50 client ready
2010-08-10 21:33:00 store 1281439970968-file.zip complete
2010-08-10 21:33:00 logout
2010-08-10 21:33:00 disconnect
2010-08-10 21:33:02 client ready
2010-08-10 21:33:11 store 1281439982234-file.zip complete
2010-08-10 21:33:11 logout
2010-08-10 21:33:11 disconnect
2010-08-10 21:33:15 client ready
2010-08-10 21:33:25 store 1281439995890-file.zip complete
2010-08-10 21:33:26 logout
2010-08-10 21:33:26 disconnect
2010-08-10 21:33:27 client ready
2010-08-10 21:33:36 store 1281440007531-file.zip complete
2010-08-10 21:33:36 logout
2010-08-10 21:33:36 disconnect
2010-08-10 21:33:37 client ready
2010-08-10 21:33:48 store 1281440017843-file.zip complete
2010-08-10 21:33:48 logout
2010-08-10 21:33:48 disconnect
2010-08-10 21:33:49 client ready
2010-08-10 21:33:59 store 1281440029781-file.zip complete
2010-08-10 21:33:59 logout
2010-08-10 21:33:59 disconnect
2010-08-10 21:34:00 client ready
2010-08-10 21:34:09 store 1281440040812-file.zip complete
2010-08-10 21:34:09 logout
2010-08-10 21:34:09 disconnect
2010-08-10 21:34:10 client ready
2010-08-10 21:34:23 store 1281440050859-file.zip complete
2010-08-10 21:34:24 logout
2010-08-10 21:34:24 disconnect
2010-08-10 21:34:25 client ready
2010-08-10 21:34:35 store 1281440065421-file.zip complete
2010-08-10 21:34:35 logout
2010-08-10 21:34:35 disconnect
Note that all of these were complete within 15 seconds, and all of the resulting files on the server are corrupt.
I have also tested without setting any timeouts and the problem still occurs.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
Commons FTP 默认为 Ascii 文件类型。当处理 ZIP 文件等二进制数据时,您需要将其设置为 Binary。
来自 http://commons.apache.org/网/api/org/apache/commons/net/ftp/FTPClient.html
您需要在发送文件之前执行
setFileType(FTP.BINARY_FILE_TYPE)
操作。Commons FTP defaults to Ascii file types. You want to set it to Binary when dealing with binary data like a ZIP file.
From http://commons.apache.org/net/api/org/apache/commons/net/ftp/FTPClient.html
You want to do
setFileType(FTP.BINARY_FILE_TYPE)
before you send the file.解决方案
我遇到了同样的问题,并通过调用
before every 方法解决了它
retrieveFile
,retrieveFileStream
,storeFile
解释
文件是已损坏,因为默认文件类型是
FTP.ASCII_FILE_TYPE
。这会导致这个问题。如果您在 Linux 上,所有字节\n\r
(Windows 文件结尾)都会更改为\n
字节。这会损坏文件。为了避免这种行为,您必须调用
ftpClient.setFileType(FTP.BINARY_FILE_TYPE)
。不幸的是,每个connect
方法都会将此设置重置回ASCII_FILE_TYPE
。就我而言,甚至可以通过
listFiles
方法重置。我猜,发生这种情况是因为我在 ftpClient 上使用passiveMode
。因此,如果您想避免麻烦,在每次文件传输之前调用
setFileType(FTP.BINARY_FILE_TYPE)
。Solution
I had the same issue and solved it by calling
before each method
retrieveFile
,retrieveFileStream
,storeFile
Explanation
File is corrupted, because default fileType is
FTP.ASCII_FILE_TYPE
. This causes the issue. If you are on linux all bytes\n\r
(windows end of file) are changed into\n
byte. And this corrupt the file.To avoid this behavior you have to call
ftpClient.setFileType(FTP.BINARY_FILE_TYPE)
. Unfortunately, this setup is reset by eachconnect
method back toASCII_FILE_TYPE
.In my case this was reset even by method
listFiles
. I guess, that this happened because I usepassiveMode
on ftpClient.So if you want to avoid troubles call
setFileType(FTP.BINARY_FILE_TYPE)
right before every file transfer.尽管指定了
二进制文件类型
,但我还是遇到了这个问题,因此我编写了代码来通过MD5
哈希验证上传的文件:MD5Checksum.java
:MD5
代码取自此处。I had this problem despite specifying
binary file type
so I wrote code to validate the uploaded file viaMD5
hashing:MD5Checksum.java
:The
MD5
code is taken from here.