Socket read timeout ignored when HttpClientConnect

Question

Below is output of jstack, watch the thread, nid=0x771d (30493). Which started several hours ago.

"taskScheduler-6" prio=10 tid=0x00007f4479e07800 nid=0x771d runnable [0x00007f446e63a000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:152)
    at java.net.SocketInputStream.read(SocketInputStream.java:122)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
    at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:554)
    at sun.security.ssl.InputRecord.read(InputRecord.java:509)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
    - locked <0x00000007601abdf0> (a java.lang.Object)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
    - locked <0x00000007601abea0> (a java.lang.Object)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory
.java:275)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:
254)
    at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:1
23)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionMa
nager.java:318)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)

    ........ // other call stack of custom codes

And the thread's CPU time is always the same, not change: (produced by top -Hp pid)

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
30493 root      20   0 3832m 1.0g  11m S   0.0  2.2   0:01.20 java

Here is the Java Code:

    RequestConfig config = RequestConfig.custom().setConnectionRequestTimeout(so_timeout_milliseconds)
                        .setConnectTimeout(so_timeout_milliseconds).setSocketTimeout(so_timeout_milliseconds).build();  // so_timeout_milliseconds = 6000
    do {
        CloseableHttpClient httpclient = HttpClients.createDefault();
        try {
            HttpGet httpget = new HttpGet(url);
            httpget.setConfig(config);
            if (headers != null) {
                for (Header header : headers) {
                    httpget.addHeader(header);
                }
            }

            CloseableHttpResponse response = httpclient.execute(httpget); // see jstack output above
            try {
                StatusLine statusLine = response.getStatusLine();
                if (statusLine.getStatusCode() == HttpStatus.SC_OK) {
                    HttpEntity entity = response.getEntity();
                    if (entity != null) {
                        InputStream instream = entity.getContent();
                        try {
                            return IOUtils.isToString(instream);
                        } catch (IOException ex) {
                            throw ex;
                        } finally {
                            instream.close();
                        }
                    }
                }
            } finally {
                response.close();
            }
        } finally {
            httpclient.close();
        }
    } while (--try_times > 0);

HttpClient version:

    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.3.5</version>
    </dependency>

1. I am not sure is this a bug in HTTPClient? If yes, what part codes make this problem?
2. Why the thread is RUNNABLE status, and the CPU time does not increase?

In my opinion, this is because of IO can not complete, so it can not occur CPU Interruption, but why the status is not SUSPEND.(I means WAITING in Java).

Answer 1

I am not sure is this a bug in HTTPClient?

No. Any such 'bug' would reside in the JVM, not JSSE, and certainly not in the HTTPClient.

If yes, what part codes make this problem?

None. See above.

Why the thread is RUNNABLE status, and the CPU time does not increase?

A Java thread being RUNNABLE means that it isn't being hindered by anything in Java such as monitors or other threads. It doesn't mean is isn't blocked from the operating system's point of view, e.g. in a blocking read, as is pbviously the case here.

In my opinion, this is because of IO can not complete, so it can not occur CPU Interruption, but why the status is not SUSPEND.

There is no such state as Thread.State.SUSPEND.

Answer 2

I have found a solution, update the httpclient to 4.3.6 and the problem is solved. This is apache JIRA. The changed codes is here.