Encrypting a large file with AES using JAVA

Question

I've tested my code with files less than this(10mb, 100mb, 500mb) and the encryption works. However, I run in to problems with files greater than 1gb. I've generated a large file (about 2gb) and I want to encrypt it with AES using JAVA, but I'm running into this error:

"Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

I've tried increasing available memory by using -Xmx8G, but no dice. Part of my code is as follows

    File selectedFile = new File("Z:\\dummy.txt");         
    Path path = Paths.get(selectedFile.getAbsolutePath());       
    byte[] toencrypt = Files.readAllBytes(path);       
    byte[] ciphertext = aesCipherForEncryption.doFinal(toencrypt);
    FileOutputStream fos = new FileOutputStream(selectedFile.getAbsolutePath());
    fos.write(ciphertext);
    fos.close();

As far as I can tell, the reason it is behaving this way, is that it is trying to read the whole file at once, encipher it, and store it into another byte array instead of buffering and streaming it in. Can anyone help me with some code tips?

I am a beginner to coding, so I don't really know much, any help will be appreciated.

Answer 1

Don't even try to read entire large files into memory. Encrypt a buffer at a time. Just do the standard copy loop with a suitably initialized CipherOutputStream wrapped around the FileOutputStream. You can use this for all files, no need to make a special case out of it. Use a buffer of 8k or more.

EDIT The 'standard copy loop' in Java is as follows:

byte[] buffer = new byte[8192];
int count;
while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

where in this case out = new CipherOutputStream(new FileOutputStream(selectedFile), cipher).

Answer 2

You can also simplify the process even further using Encryptor4j that I have authored: https://github.com/martinwithaar/Encryptor4j

File srcFile = new File("original.zip");
File destFile = new File("original.zip.encrypted");
String password = "mysupersecretpassword";
FileEncryptor fe = new FileEncryptor(password);
fe.encrypt(srcFile, destFile);

This library uses streaming encryption so it will not cause OutOfMemoryError even with large files. Also, instead of using passwords you can use your own Key as well.

Check out the example on the Github page here: https://github.com/martinwithaar/Encryptor4j#file-encryption