FileChannel.transferTo for large file in windows

问题:

Using Java NIO use can copy file faster. I found two kind of method mainly over internet to do this job.

public static void copyFile(File sourceFile, File destinationFile) throws IOException {
    if (!destinationFile.exists()) {
        destinationFile.createNewFile();
    }

    FileChannel source = null;
    FileChannel destination = null;
    try {
        source = new FileInputStream(sourceFile).getChannel();
        destination = new FileOutputStream(destinationFile).getChannel();
        destination.transferFrom(source, 0, source.size());
    } finally {
        if (source != null) {
            source.close();
        }
        if (destination != null) {
            destination.close();
        }
    }
}

In 20 very useful Java code snippets for Java Developers I found a different comment and trick:

public static void fileCopy(File in, File out) throws IOException {
    FileChannel inChannel = new FileInputStream(in).getChannel();
    FileChannel outChannel = new FileOutputStream(out).getChannel();
    try {
        // inChannel.transferTo(0, inChannel.size(), outChannel); // original -- apparently has trouble copying large files on Windows
        // magic number for Windows, (64Mb - 32Kb)
        int maxCount = (64 * 1024 * 1024) - (32 * 1024);
        long size = inChannel.size();
        long position = 0;
        while (position < size) {
            position += inChannel.transferTo(position, maxCount, outChannel);
        }
    } finally {
        if (inChannel != null) {
            inChannel.close();
        }
        if (outChannel != null) {
            outChannel.close();
        }
    }
}

But I didn't find or understand what is meaning of

"magic number for Windows, (64Mb - 32Kb)"

It says that inChannel.transferTo(0, inChannel.size(), outChannel) has problem in windows, is 32768 (= (64 * 1024 * 1024) - (32 * 1024)) byte is optimum for this method.

回答1:

Windows has a hard limit on the maximum transfer size, and if you exceed it you get a runtime exception. So you need to tune. The second version you give is superior because it doesn't assume the file was transferred completely with one transferTo() call, which agrees with the Javadoc.

Setting the transfer size more than about 1MB is pretty pointless anyway.

EDIT Your second version has a flaw. You should decrement size by the amount transferred each time. It should be more like:

while (position < size) {
    long count = inChannel.transferTo(position, size, outChannel);
    if (count > 0)
    {
        position += count;
        size-= count;
    }
}

回答2:

I have read that it is for compatibility with the Windows 2000 operating system.

Source: http://www.rgagnon.com/javadetails/java-0064.html

Quote: In win2000, the transferTo() does not transfer files > than 2^31-1 bytes. it throws an exception of "java.io.IOException: Insufficient system resources exist to complete the requested service is thrown." The workaround is to copy in a loop 64Mb each time until there is no more data.

回答3:

There appears to be anecdotal evidence that attempts to transfer more than 64MB at a time on certain Windows versions results in a slow copy. Hence the check: this appears to be the result of some detail of the underlying native code that implements the transferTo operation on Windows.