Android: Too many open files error

Question

I have the following operation which runs every 3 seconds.
Basically it downloads a file from a server and save it into a local file every 3 seconds.
The following code does the job for a while.

public class DownloadTask extends AsyncTask<String, Void, String>{

    @Override
    protected String doInBackground(String... params) {
        downloadCommandFile( eventUrl);
        return null;
    }


}

private void downloadCommandFile(String dlUrl){
    int count;
    try {
        URL url = new URL( dlUrl );
        NetUtils.trustAllHosts();
        HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
        con.setDoInput(true);
        con.setDoOutput(true);
        con.connect();
        int fileSize = con.getContentLength();
        Log.d(TAG, "Download file size = " + fileSize );
        InputStream is = url.openStream();
        String dir = Environment.getExternalStorageDirectory() + Utils.DL_DIRECTORY;
        File file = new File( dir );
        if( !file.exists() ){
            file.mkdir();
        }

        FileOutputStream fos = new FileOutputStream(file + Utils.DL_FILE);
        byte data[] = new byte[1024];
        long total = 0;

        while( (count = is.read(data)) != -1 ){
            total += count;
            fos.write(data, 0, count);
        }

        is.close();
        fos.close();
        con.disconnect(); // close connection


    } catch (Exception e) {
        Log.e(TAG, "DOWNLOAD ERROR = " + e.toString() );
    }

}

Everything works fine, but if I leave it running for 5 to 10 minutes I get the following error.

06-04 19:40:40.872: E/NativeCrypto(6320): AppData::create pipe(2) failed: Too many open files 06-04 19:40:40.892: E/NativeCrypto(6320): AppData::create pipe(2) failed: Too many open files 06-04 19:40:40.892: E/EventService(6320): DOWNLOAD ERROR = javax.net.ssl.SSLException: Unable to create application data

I have been doing some researches for the last 2 days.
There are suggestions that they are many connections open, like this one https://stackoverflow.com/a/13990490/1503155 but still I can not figure out what's the problem.
Any ideas what may cause the problem?
Thanks in advance.

Answer 1

I think you get this error because you have too many files open at the same times, meaning that you have too many async tasks running in the same time (each async task opens a file), which makes sense if you say that you run a new one every 3 seconds.

You should try to limit the number of async task running in the same time using a thread pool executor.

Answer 2

Try using OkHttp Instead.

Your issue isn't with too many threads, although that's what's causing your issue to surface.

As @stdout mentioned in the comments, AsyncTask already runs in a threadpool that is common and shared amongst all AsyncTasks, unless you specify otherwise. The issue here is that the file descriptors are not being closed properly and in time.

The issue is that your file descriptors are not being closed fast enough.

I struggled with this for hours/days/weeks, doing everything you should like setting small read/connect timeouts and using a finally block to close out connections, input streams, output streams, etc. But we never found a working solution. It seemed like HttpsUrlConnection was flawed in some way.

So we tried OkHttp as a drop-in replacement for HttpsUrlConnection and voila! It worked out of the box.

So, if you're struggling with this and are having a really hard time fixing it, I suggest you try using OkHttp as well.

Here are the basics:

Once you get the Maven dependency added, you can do something like the following to download a file:

OkHttpClient okHttpClient = new OkHttpClient.Builder().build();

OutputStream output = null;

try {
  Request request   = new Request.Builder().url( download_url ).build();
  Response response = okHttpClient.newCall( request ).execute();

  if ( !response.isSuccessful() ) {
    throw new FileNotFoundException();
  }

  output = new FileOutputStream( output_path );

  output.write( response.body().bytes() );
}
finally {
  // Ensure streams are closed, even if there's an exception.
  if ( output != null ) output.flush();
  if ( output != null ) output.close();
}

Switching to OkHttp instantly fixed our leaked file descriptor issue so it's worth trying if you're stuck, even at the expense of adding another library dependency.