java.io.IOException: Invalid argument

Question

I have a web application running in cluster mode with a load balancer. It consists in two tomcats (T1, and T2) addressing only one DB. T2 is nfs mounted to T1. This is the only dofference between both nodes.

I have a java method generating some files. If the request runs on T1 there is no problem but if the request is running on node 2 I get an exception as follows:

java.io.IOException: Invalid argument
        at java.io.FileOutputStream.close0(Native Method)
        at java.io.FileOutputStream.close(FileOutputStream.java:279)

The corresponding code is as follows:

for (int i = 0; i < dataFileList.size(); i++) {
    outputFileName = outputFolder + fileNameList.get(i);
    FileOutputStream fileOut = new FileOutputStream(outputFileName);                        
    fileOut.write(dataFileList.get(i), 0, dataFileList.get(i).length);
    fileOut.flush();
    fileOut.close();
}

The exception appears at the fileOut.close()

Any hint?

Luis

Answer 1

How large do dataFileList and fileNameList get? You could be running out of file descriptors. It's odd that it happens on close(), though.

Answer 2

Finally I found the reason. First I've notices that NOT always this exception comes at the same point.

Sometimes was a java.io.IOException: Invalid argument at java.io.FileOutputStream.close0(Native Method) at java.io.FileOutputStream.close(FileOutputStream.java:279) ^^^^^

and sometimes was

java.io.IOException: Invalid argument
    at java.io.FileOutputStream.writeBytes(Native Method)
    at java.io.FileOutputStream.write(FileOutputStream.java:260)

Therefore the problem is NOT a java problem. Not even a NFS problem. The problem is the underlying File System type which is an DRBD file system.

Testing at a shell to write across the nodes works if one is writing a small file. I.e:

at the nfs mounted node

cd /tmp
date > /shared/path-to-some-not-mounted-dir/today

will work

but

cat myBigFile > /shared/path-to-some-not-mounted-dir/today

will deliver the following error

cat: write error: Invalid argument

Therefore the solution is to use other type of file system, gfs for example.

Answer 3

Setting this line in the .profile resolved the issue:

ulimit –n 2048