How to keep HttpClient Connection Keep-Alive?

Question

I am working HttpClient POST method. I need to create HttpClient once and should use Keep Alive Connection. But I think in my case , its establishing a new connection every time.

So, i need to use a Keep Alive connection for HttpClient.

Here is my code snippet any help would be appreciated lot.

ClientConnectionManager mgr = httpclient_recv.getConnectionManager();
    hp = httpclient_recv.getParams();
    httpclient_recv = new DefaultHttpClient(
    new ThreadSafeClientConnManager(hp,mgr.getSchemeRegistry()), hp);

    while (true) {

        try {

            java.util.logging.Logger.getLogger("org.apache.http.wire")
                    .setLevel(java.util.logging.Level.FINER);
            java.util.logging.Logger.getLogger("org.apache.http.headers")
                    .setLevel(java.util.logging.Level.FINER);

            System.setProperty("org.apache.commons.logging.Log",
                    "org.apache.commons.logging.impl.SimpleLog");
            System.setProperty(
                    "org.apache.commons.logging.simplelog.showdatetime",
                    "true");
            System.setProperty(
                    "org.apache.commons.logging.simplelog.log.httpclient.wire",
                    "debug");
            System.setProperty(
                    "org.apache.commons.logging.simplelog.log.org.apache.http",
                    "debug");
            System.setProperty(
                    "org.apache.commons.logging.simplelog.log.org.apache.http.headers",
                    "debug");

            ByteArrayEntity bae = new ByteArrayEntity(byteData);
            bae.setContentType(new BasicHeader(HTTP.CONTENT_TYPE,
                    "binary/octet-stream"));

            httppost_recv.setHeader(HTTP.CONN_DIRECTIVE,HTTP.CONN_KEEP_ALIVE);
            httppost_recv.setEntity(bae);



            {
                System.out.println("res b4 response");
                 response_recv = httpclient_recv
                        .execute(httppost_recv);
                 response_recv.getEntity().consumeContent();
                System.out.println("res a4 response");
                if (response_recv != null) {
                    byteArray = EntityUtils.toByteArray(response_recv
                            .getEntity());
                    playing  = true;
                }
                        }
                 }

and also logcat logs is:

12-03 10:07:29.466: I/System.out(1529): res b4 response
12-03 10:07:29.646: D/org.apache.http.wire(1529): >> "POST /ping HTTP/1.1[EOL]"
12-03 10:07:29.666: D/org.apache.http.wire(1529): >> "Connection: Keep-Alive[EOL]"
12-03 10:07:29.686: D/org.apache.http.wire(1529): >> "Content-Length: 1[EOL]"
12-03 10:07:29.705: D/org.apache.http.wire(1529): >> "Content-Type: binary/octet-stream[EOL]"
12-03 10:07:29.716: D/org.apache.http.wire(1529): >> "Host: 192.168.1.36[EOL]"
12-03 10:07:29.725: D/org.apache.http.wire(1529): >> "User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)[EOL]"
12-03 10:07:29.736: D/org.apache.http.wire(1529): >> "[EOL]"
12-03 10:07:29.746: D/org.apache.http.headers(1529): >> POST /ping HTTP/1.1
12-03 10:07:29.746: D/org.apache.http.headers(1529): >> Connection: Keep-Alive
12-03 10:07:29.756: D/org.apache.http.headers(1529): >> Content-Length: 1
12-03 10:07:29.756: D/org.apache.http.headers(1529): >> Content-Type: binary/octet-stream
12-03 10:07:29.765: D/org.apache.http.headers(1529): >> Host: 192.168.1.36
12-03 10:07:29.765: D/org.apache.http.headers(1529): >> User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
12-03 10:07:29.776: D/org.apache.http.wire(1529): >> "[0x0]"
12-03 10:07:29.796: D/org.apache.http.wire(1529): << "HTTP/1.1 200 OK[EOL]"
12-03 10:07:29.805: D/org.apache.http.wire(1529): << "Server: gSOAP/2.8[EOL]"
12-03 10:07:29.816: D/org.apache.http.wire(1529): << "Content-Type: binary/octet-stream[EOL]"
12-03 10:07:29.826: D/org.apache.http.wire(1529): << "Content-Length: 2048[EOL]"
12-03 10:07:29.836: D/org.apache.http.wire(1529): << **"Connection: close[EOL]"**
12-03 10:07:29.887: D/org.apache.http.headers(1529): << HTTP/1.1 200 OK
12-03 10:07:29.896: D/org.apache.http.headers(1529): << Server: gSOAP/2.8
12-03 10:07:29.896: D/org.apache.http.headers(1529): << Content-Type: binary/octet-stream
12-03 10:07:29.906: D/org.apache.http.headers(1529): << Content-Length: 2048
12-03 10:07:29.906: D/org.apache.http.headers(1529): << **Connection: close**
12-03 10:07:29.916: I/System.out(1529): res a4 response

Answer 1

From HTTP 1.1, keep-alive is enabled by default. You would need to close the connection if you don't want it to be reused explicitly when dealing with HTTP 1.1.

For 1.0, an header is what you set for this "Connection: Keep-alive" This only intimates the server that you want to reuse the connection. When under stress or for other reasons, server might choose to act differently as explained below.

For most purposes most answers here are correct, where in you add keep alive header and it works well. The following explanation is for the scenarios where you did that but it still does not work.

---

The server side issues

Normally an answer would focus at a setup when server would behave normally, but this is not completely true. Servers (like Rudra) are built to behave differently in different situations. Keep-alive comes with a number of requests the server would serve you for before dropping your connection, this is there to allow service to others as well so in case of high load, some servers might resort to reducing the no of keep-alive requests served for each new connection.

There is also a timeout set from the last request received, which would eventually lead to disconnection if no further requests are made in that window of time. Few modern servers might alter this based on the capacity they have at the moment or drop it to 0 in panic conditions making keep-alive meaningless. So if the server you are trying to connect with is going through any of such (race,panic) conditions it might choose to discard your request.

---

The client side issues

For the documentation purpose. From hc.apache.org :

HttpClient always does its best to reuse connections. Connection persistence is enabled by default and requires no configuration. Under some situations this can lead to leaked connections and therefore lost resources. The easiest way to disable connection persistence is to provide or extend a connection manager that force-closes connections upon release in the releaseConnection method.

HttpClient offers these (read:trivial) things Out Of The Box. But still there are other things that are offered by Apache that you can add to improve it's performance.

ConnectionManager(s) for example can be customized for HttpClient.

So the thing that can block keep-alive/connection persistence is the connection manager that you might be using (this is not true in your case, but it might be true in several other cases). This might be a totally unknown/abstract fact for you if you are getting the Client object for making the calls from some API. An example of how this can be customized has been listed below (from Apache connection management documentation)

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
// Increase max total connection to 200
cm.setMaxTotal(200);
// Increase default max connection per route to 20
cm.setDefaultMaxPerRoute(20);
// Increase max connections for localhost:80 to 50
HttpHost localhost = new HttpHost("locahost", 80);
cm.setMaxPerRoute(new HttpRoute(localhost), 50);

CloseableHttpClient httpClient = HttpClients.custom()
    .setConnectionManager(cm)
    .build();

(please refer Apache documentation on connection management for more details)

If you face this problem, try out without a CM or create your own HttpClient object. It's not necessary to use a CM for multiple connections as well. The inbuilt CM is fair enough. If you see a performance loss you can write your own connection manager.

In your case however, your server is not very supportive it might not honour Keep-alive at all, even if you have those headers and what not. You would need to check for timeout header in response, on sending keep-alive in a new request to the server to establish that server is complaint.

Answer 2

10:07:29.746: D/org.apache.http.headers(1529): >> Connection: Keep-Alive

You are requesting keepalive.

10:07:29.836: D/org.apache.http.wire(1529): << "Connection: close[EOL]"

The server is refusing it.

Nothing you can do about that at your end.

Answer 3

Why don't you just use the same client for all your requests???

I've been working on an app that needs to request a lot of data to many web services, and for that I only use one static client and works perfect!

I have made a class that returns the HttpClient and another class that manage all my requests (POST and GET), nice and easy ;)