I have a web application running in cluster mode with a load balancer.
It consists in two tomcats (T1, and T2) addressing only one DB.
T2 is nfs mounted to T1. This is the only dofference between both nodes.
I have a java method generating some files. If the request
runs on T1 there is no problem but if the request is running on node 2
I get an exception as follows:
java.io.IOException: Invalid argument
at java.io.FileOutputStream.close0(Native Method)
at java.io.FileOutputStream.close(FileOutputStream.java:279)
The corresponding code is as follows:
for (int i = 0; i < dataFileList.size(); i++) {
outputFileName = outputFolder + fileNameList.get(i);
FileOutputStream fileOut = new FileOutputStream(outputFileName);
fileOut.write(dataFileList.get(i), 0, dataFileList.get(i).length);
fileOut.flush();
fileOut.close();
}
The exception appears at the fileOut.close()
Any hint?
Luis
Setting this line in the .profile
resolved the issue:
ulimit –n 2048
How large do dataFileList and fileNameList get? You could be running out of file descriptors. It's odd that it happens on close(), though.
Finally I found the reason.
First I've notices that NOT always this exception comes
at the same point.
Sometimes was a
java.io.IOException: Invalid argument
at java.io.FileOutputStream.close0(Native Method)
at java.io.FileOutputStream.close(FileOutputStream.java:279)
^^^^^
and sometimes was
java.io.IOException: Invalid argument
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
Therefore the problem is NOT a java problem. Not even a NFS problem.
The problem is the underlying File System type which is an DRBD
file system.
Testing at a shell to write across the nodes works if one is writing a small
file. I.e:
at the nfs mounted node
cd /tmp
date > /shared/path-to-some-not-mounted-dir/today
will work
but
cat myBigFile > /shared/path-to-some-not-mounted-dir/today
will deliver the following error
cat: write error: Invalid argument
Therefore the solution is to use other type of file system, gfs for example.