Safe use of HttpURLConnection

Question

When using HttpURLConnection does the InputStream need to be closed if we do not 'get' and use it?

i.e. is this safe?

HttpURLConnection conn = (HttpURLConnection) uri.getURI().toURL().openConnection();
conn.connect();
// check for content type I don't care about
if (conn.getContentType.equals("image/gif") return; 
// get stream and read from it
InputStream is = conn.getInputStream();
try {
    // read from is
} finally {
    is.close();
}

Secondly, is it safe to close an InputStream before all of it's content has been fully read?

Is there a risk of leaving the underlying socket in ESTABLISHED or even CLOSE_WAIT state?

Answer 1

When using HttpURLConnection does the InputStream need to be closed if we do not 'get' and use it?

Yes, it always needs to be closed.

i.e. is this safe?

Not 100%, you run the risk of getting a NPE. Safer is:

InputStream is = null;
try {
    is = conn.getInputStream()
    // read from is
} finally {
    if (is != null) {
        is.close();
    }
}

Answer 2

is it safe to close an InputStream before all of it's content has been read

You need to read all of the data in the input stream before you close it so that the underlying TCP connection gets cached. I have read that it should not be required in latest Java, but it was always mandated to read the whole response for connection re-use.

Check this post: keep-alive in java6

Answer 3

According to http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html and OpenJDK source code.

(When keepAlive == true)

If client called HttpURLConnection.getInputSteam().close(), the later call to HttpURLConnection.disconnect() will NOT close the Socket. i.e. The Socket is reused (cached)

If client does not call close(), call disconnect() will close the InputStream and close the Socket.

So in order to reuse the Socket, just call InputStream.close(). Do not call HttpURLConnection.disconnect().

Answer 4

You also have to close error stream if the HTTP request fails (anything but 200):

try {
  ...
}
catch (IOException e) {
  connection.getErrorStream().close();
}

If you don't do it, all requests that don't return 200 (e.g. timeout) will leak one socket.

Answer 5

Here is some information regarding the keep-alive cache. All of this information pertains Java 6, but is probably also accurate for many prior and later versions.

From what I can tell, the code boils down to:

1. If the remote server sends a "Keep-Alive" header with a "timeout" value that can be parsed as a positive integer, that number of seconds is used for the timeout.
2. If the remote server sends a "Keep-Alive" header but it doesn't have a "timeout" value that can be parsed as a positive integer and "usingProxy" is true, then the timeout is 60 seconds.
3. In all other cases, the timeout is 5 seconds.

This logic is split between two places: around line 725 of sun.net.www.http.HttpClient (in the "parseHTTPHeader" method), and around line 120 of sun.net.www.http.KeepAliveCache (in the "put" method).

---

So, there are two ways to control the timeout period:

1. Control the remote server and configure it to send a Keep-Alive header with the proper timeout field
2. Modify the JDK source code and build your own.

One would think that it would be possible to change the apparently arbitrary five-second default without recompiling internal JDK classes, but it isn't. A bug was filed in 2005 requesting this ability, but Sun refused to provide it.

Answer 6

If you really want to make sure that the connection is close you should call conn.disconnect().

The open connections you observed are because of the HTTP 1.1 connection keep alive feature (also known as HTTP Persistent Connections). If the server supports HTTP 1.1 and does not send a Connection: close in the response header Java does not immediately close the underlaying TCP connection when you close the input stream. Instead it keeps it open and tries to reuse it for the next HTTP request to the same server.

If you don't want this behaviour at all you can set the system property http.keepAlive to false:

System.setProperty("http.keepAlive","false");