Java开发FTP功能的apache工具包，小心使用为妙

项目是一个报表系统，使用apache-commons-net网络工具包实现文件上传与下载。实际测试中报表数量比较小，没有发现大问题，并且开发时也是本地开发，本地FTP服务器处理。后来测试发现，该处理使用的是单字节不刷新读取模式，一个文件如果几百M,读入缓冲字节流的数据也会在内存中占用几百个M,这一点实际开发测试可以断点调试，使用VisualVM监控即可。

查看FTP上传下载源码，发现可以再FTPClient注入缓冲字节大小，实际测试8*1024性能比较好：

ftpClient.changeWorkingDirectory(new String(path.getBytes("utf-8"), "ISO8859-1"));
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
// long start = System.currentTimeMillis();
ftpClient.setBufferSize(1024 * 8);
ftpClient.storeFile(new String(fileName.getBytes("utf-8"), "ISO8859-1"), input);
// System.out.println(System.currentTimeMillis()-start);

设置后，发现效率并没有显著提高，查看storeFile文件上传源码了，里面的核心处理还记基于流的读写，如下所示：

protected boolean _storeFile(String command,
String remote,
InputStream local) throws IOException {
Socket socket = _openDataConnection_(command, remote);

if (socket == null) {
return false;
}

OutputStream output = getBufferedOutputStream(socket.getOutputStream());

if (__fileType == ASCII_FILE_TYPE) {
output = new ToNetASCIIOutputStream(output);
}

CSL csl = null;
if (__controlKeepAliveTimeout > 0) {
csl = new CSL(this, __controlKeepAliveTimeout, __controlKeepAliveReplyTimeout);
}

// Treat everything else as binary for now
try {
Util.copyStream(local, output, getBufferSize(), CopyStreamEvent.UNKNOWN_STREAM_SIZE,
__mergeListeners(csl), false);
} catch (IOException e) {
Util.closeQuietly(socket); // ignore close errors here
if (csl != null) {
csl.cleanUp(); // fetch any outstanding keepalive replies
}
throw e;
}

output.close(); // ensure the file is fully written
socket.close(); // done writing the file
if (csl != null) {
csl.cleanUp(); // fetch any outstanding keepalive replies
}
// Get the transfer response
boolean ok = completePendingCommand();
return ok;
}

其中的

Util.copyStream

是主要的处理方法，这里的最后一个参数为false,表示是否实时flush,但是这个方法的属性设置 包中没有外部接口，坑爹了。进去看一看这个上传的源码：

public static final long copyStream(InputStream source, OutputStream dest,
int bufferSize, long streamSize,
CopyStreamListener listener,
boolean flush)
throws CopyStreamException
{
int bytes;
long total = 0;
<strong>byte[] buffer = new byte[bufferSize >= 0 ? bufferSize : DEFAULT_COPY_BUFFER_SIZE];</strong>

try
{
while ((bytes = source.read(buffer)) != -1)
{
// Technically, some read(byte[]) methods may return 0 and we cannot
// accept that as an indication of EOF.

if (bytes == 0)
{
bytes = source.read();
if (bytes < 0) {
break;
}
dest.write(bytes);
<strong>if(flush) {
dest.flush();
}</strong>
++total;
if (listener != null) {
listener.bytesTransferred(total, 1, streamSize);
}
continue;
}

dest.write(buffer, 0, bytes);
<strong>if(flush) {
dest.flush();
}</strong>
total += bytes;
if (listener != null) {
listener.bytesTransferred(total, bytes, streamSize);
}
}
}
catch (IOException e)
{
throw new CopyStreamException("IOException caught while copying.",
total, e);
}

return total;
}

看看加粗的代码，一个是缓冲流大小，一个是是否刷新流。找了半天flush没有注入接口，没办法，最后想通过继承的方式，在应用中实现这个类，并重写方法，最后发现，难度有点大，依赖属性太多。最好把FTPClient直接拉到自己应用中，全部复制吧，如下：

public class<strong> FTPClient extends org.apache.commons.net.ftp.FTPClient</strong> {

/**
* The system property ({@value} ) which can be used to override the system type.<br/>
* If defined, the value will be used to create any automatically created parsers.
*
* @since 3.0
*/
public static final String FTP_SYSTEM_TYPE = "org.apache.commons.net.ftp.systemType";

/**
* The system property ({@value} ) which can be used as the default system type.<br/>
* If defined, the value will be used if the SYST command fails.
*

其它代码照搬吧。不管怎么样，最后的传输效率大大提高，而且是时刻刷新，减轻了服务器压力。

可能对此还不是深入理解，遗漏出望指教。