received handshake warning: unrecognized_name

Question

Is there a way to deal in Java 11 with a "javax.net.ssl.SSLHandshakeException: received handshake warning: unrecognized_name" without disabling SNI system wide by using

System.setProperty("jsse.enableSNIExtension", "false")

Using this system property would cause any following request to a host depending on SNI to fail. So basically I do need a per request solution.

Very specific: I am trying to get the content from the site https://www.minervamedica.it which seems to have issues for Java > 8.

I did try e.g. this approach: https://javabreaks.blogspot.com/2015/12/java-ssl-handshake-with-server-name.html

        final TrustManager[] trustAllCerts = new TrustManager[] { new X509ExtendedTrustManager() {
            @Override
            public void checkClientTrusted(final X509Certificate[] x509Certificates, final String s)
                    throws CertificateException {
            }

            @Override
            public void checkServerTrusted(final X509Certificate[] x509Certificates, final String s)
                    throws CertificateException {
            }

            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null;
            }

            @Override
            public void checkClientTrusted(final X509Certificate[] x509Certificates, final String s,
                    final Socket socket) throws CertificateException {
            }

            @Override
            public void checkServerTrusted(final X509Certificate[] x509Certificates, final String s,
                    final Socket socket) throws CertificateException {
            }

            @Override
            public void checkClientTrusted(final X509Certificate[] x509Certificates, final String s,
                    final SSLEngine sslEngine) throws CertificateException {
            }

            @Override
            public void checkServerTrusted(final X509Certificate[] x509Certificates, final String s,
                    final SSLEngine sslEngine) throws CertificateException {
            }
        } };

        final SSLContext sslContext = SSLContext.getInstance("SSL");
        sslContext.init(null, trustAllCerts, new java.security.SecureRandom());

        URL url = new URL("https://www.minervamedica.it");
        SSLParameters sslParameters = new SSLParameters();
        List<SNIServerName> sniHostNames = new ArrayList<>();
        sniHostNames.add(new SNIHostName(url.getHost()));
//      sniHostNames.add(new SNIHostName("minervamedica.it"));
        sslParameters.setServerNames(sniHostNames);
        SSLSocketFactory wrappedSSLSocketFactory = new SSLSocketFactoryWrapper(sslContext.getSocketFactory(), sslParameters);

        HttpsURLConnection connection = (HttpsURLConnection) url
                .openConnection();
        connection.setSSLSocketFactory(wrappedSSLSocketFactory);
        connection.setDoOutput(true);
        connection.setRequestMethod("GET");
        System.out.print(connection.getResponseCode());

SSLSocketFactoryWrapper

import java.io.IOException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;

public class SSLSocketFactoryWrapper extends SSLSocketFactory {

    private final SSLSocketFactory wrappedFactory;
    private final SSLParameters sslParameters;

    public SSLSocketFactoryWrapper(SSLSocketFactory factory, SSLParameters sslParameters) {
        this.wrappedFactory = factory;
        this.sslParameters = sslParameters;
    }

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port);
        setParameters(socket);
        return socket;
    }

    @Override
    public Socket createSocket(String host, int port, InetAddress localHost, int localPort)
            throws IOException, UnknownHostException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port, localHost, localPort);
        setParameters(socket);
        return socket;
    }


    @Override
    public Socket createSocket(InetAddress host, int port) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port);
        setParameters(socket);
        return socket;
    }

    @Override
    public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(address, port, localAddress, localPort);
        setParameters(socket);
        return socket;

    }

    @Override
    public Socket createSocket() throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket();
        setParameters(socket);
        return socket;
    }

    @Override
    public String[] getDefaultCipherSuites() {
        return wrappedFactory.getDefaultCipherSuites();
    }

    @Override
    public String[] getSupportedCipherSuites() {
        return wrappedFactory.getSupportedCipherSuites();
    }

    @Override
    public Socket createSocket(Socket s, String host, int port, boolean autoClose) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(s, host, port, autoClose);
        setParameters(socket);
        return socket;
    }

    private void setParameters(SSLSocket socket) {
        socket.setSSLParameters(sslParameters);
    }

}

Edit 30.04.2019:

Also not working on Java 11 is something like:

URL url = new URL("https://www.minervamedica.it");
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setHostnameVerifier((s, sslSession) -> true);
System.out.println(new String(conn.getInputStream().readAllBytes()));

This also results in a exception with "received handshake warning: unrecognized_name", despite using a custom HostnameVerifier which returns always true.

---

Edit 02.05.2019

It is apparently a misconfigured server (see below).

openssl s_client -servername www.minervamedica.it -connect www.minervamedica.it:443 -state

reveals

CONNECTED(00000003)
SSL_connect:before SSL initialization
SSL_connect:SSLv3/TLS write client hello
SSL3 alert read:warning:unrecognized name
SSL_connect:SSLv3/TLS write client hello
SSL_connect:SSLv3/TLS read server hello
depth=0 CN = minervamedica.it
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = minervamedica.it
verify error:num=21:unable to verify the first certificate
verify return:1
SSL_connect:SSLv3/TLS read server certificate
SSL_connect:SSLv3/TLS read server key exchange
SSL_connect:SSLv3/TLS read server done
SSL_connect:SSLv3/TLS write client key exchange
SSL_connect:SSLv3/TLS write change cipher spec
SSL_connect:SSLv3/TLS write finished
SSL_connect:SSLv3/TLS write finished
SSL_connect:SSLv3/TLS read server session ticket
SSL_connect:SSLv3/TLS read change cipher spec
SSL_connect:SSLv3/TLS read finished
---
Certificate chain
 0 s:/CN=minervamedica.it
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFbTCCBFWgAwIBAgISA7svnlD9ZgJAww8LaYbVvgQ2MA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xOTA1MDIwNjAyNDlaFw0x
OTA3MzEwNjAyNDlaMBsxGTAXBgNVBAMTEG1pbmVydmFtZWRpY2EuaXQwggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCuJUYVVxEhkHmGlPBT/zKZ6NrGuKls
pYSeeJsa8mMWwCf5b0ZOe8jS8w36EHvTUbHHOqd63zXjsFDkt14SmNunogSCSYvq
8+UmHPudv2q4ygPLY728bU5YpXVXaBh6hcJmfckCs0WnxLbPFC3rJdlC77syDbpi
O/fX5XY7cmzB7gCH3MmKltGzk3oQDYst4IIFZZV11Hk1VVDJ7MAb23E4PINKEJwJ
5IqFJRjko3nVvKEY+FVv0Bl4N7PN8xl9M+Xw4Bcp8sUaGmgbRSPAbPj2S1LWoRq+
dUFyqsmmks0YsdMbuRfkjWUuJ5h0MUtpW0yCbJIFtUgEysJzREgfTX5NAgMBAAGj
ggJ6MIICdjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG                                                          
AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFLjAn5EGedAmqtkEk2OHsrJw                                                          
cOW6MB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZFZe/zqOyhMG8GCCsGAQUFBwEB                                                          
BGMwYTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AuaW50LXgzLmxldHNlbmNyeXB0
Lm9yZzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQuaW50LXgzLmxldHNlbmNyeXB0
Lm9yZy8wLwYDVR0RBCgwJoISKi5taW5lcnZhbWVkaWNhLml0ghBtaW5lcnZhbWVk
aWNhLml0MEwGA1UdIARFMEMwCAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYI
KwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0c2VuY3J5cHQub3JnMIIBBQYKKwYBBAHW
eQIEAgSB9gSB8wDxAHYA4mlLribo6UAJ6IYbtjuD1D7n/nSI+6SPKJMBnd3x2/4A
AAFqd1pUHgAABAMARzBFAiEAsPuYjgg2alECcOV36YTwGawdYOi2dcPuTUHN7FL2
kJUCIGnckeJZKe1Xb9tJA3YkuxptbOFlHEUBAaDQEiz49CMJAHcAKTxRllTIOWW6
qlD8WAfUt2+/WHopctykwwz05UVH9HgAAAFqd1pUBQAABAMASDBGAiEAx0NQhPfK
FpAPHJ8ZU6BxLGl4gXMND4FxuMVsGb+pfxYCIQDv0lsXnvPmEIQdCMero8IyjrYk
L8K9f1zVbSAFn/6PxzANBgkqhkiG9w0BAQsFAAOCAQEAOf0IE45r4ytrFtFXrVMY
ATpt/UcTgJvqgapg4KsQSr4k007MZtxeALRn6B5KdekGhhKzIlHz6O/JD4+95Btv
mempZgo166Nr4sf4UMfNsENNqUX1jgT2i74Ss6058t6YtTanuNdrokL/mMxSynIt
5O49srcpEwhTvIaeKq84DLd6Es9OcBuRAJZCEw/SGtLypkC0PSSHayuGvJjssDX6
RB+CkftpKJC9c6M+5e1fXjMHDHrUPukS467vs0Ky5jK0ZzFna7NAQtip7XY1TyAi
b6AdnLLGbKt6DIb6eOnTf5aIMatyTCRAVcVKSSqdtAhX0aS33/iXFPbApF3E0GyQ
gA==
-----END CERTIFICATE-----
subject=/CN=minervamedica.it
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2106 bytes and written 331 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 008EB1ED9FEA6E2FE477231DB86F669C3C44792C2A02A80FCBB955186E141C86
    Session-ID-ctx: 
    Master-Key: 7745BDCCE5390A15586866EBA311DDCA90BD75AA7D91D5825A23DEF83B6C88CD56BFFC53ECDBA67271BFD8AB720D8522
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - ef 8d 8c e9 3a 3c 79 d9-74 cd 3f f2 f2 d7 55 4a   ....:<y.t.?...UJ
    0010 - a0 45 9b a9 f0 22 2b 13-19 8c d5 8b be 57 be 6c   .E..."+......W.l
    0020 - 38 ab f6 92 21 a4 ef 93-20 bf c2 f9 53 ee df 96   8...!... ...S...
    0030 - a0 68 fe ab ff 5e e0 85-c7 7f 2f 4d f7 b6 c6 7f   .h...^..../M....
    0040 - 6b d1 42 ff ab 96 eb 1e-1b ef 98 f4 68 bb ee 45   k.B.........h..E
    0050 - 0a f1 0b 4e 88 41 95 fc-b9 a2 9a 93 38 21 bd 6e   ...N.A......8!.n
    0060 - 84 9d 54 d7 27 d5 c9 94-87 b6 03 29 5d c7 87 07   ..T.'......)]...
    0070 - 99 ee c3 27 5a 57 02 19-66 fe 89 43 d5 b6 bb 90   ...'ZW..f..C....
    0080 - 4c ce fb 3c da 91 75 75-e7 99 a4 87 7c 92 57 d3   L..<..uu....|.W.
    0090 - f3 5b 5d 62 45 82 27 97-d8 8a 0d c3 e1 f3 7b b8   .[]bE.'.......{.
    00a0 - fd 28 1f 59 7f 74 a2 29-ae 11 c4 b4 ef c0 65 23   .(.Y.t.)......e#
    00b0 - 48 6e c2 a3 fc fa cf 05-56 f0 ce 2c 36 54 02 b9   Hn......V..,6T..
    00c0 - a2 12 ef 86 cb 8d bd ae-b0 ff 4c 0c a2 72 36 11   ..........L..r6.

    Start Time: 1556795487
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
---

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>
SSL3 alert read:warning:close notify
closed
SSL3 alert write:warning:close notify

But nevertheless, Java 11 seems not be able to deal with the situation as prior versions did. Using a custom HostnameVerifier seems to get totally ignored. Is this a Java bug?

Update 11.05.2019

A bug report has been opened: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8223677

Edit 12.05.2019

As dave_thompson_085 correctly points out, the behaviour is the same in Java 8 and Java 11: the exception is alwas thrown.

The only way to get around this issue up to Java 8 was using org.apache.http, but this does not work on newer Java versions like Java 11:

package test;

import java.io.IOException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.ssl.SSLContext;

import org.apache.http.HttpEntity;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.HttpConnectionFactory;
import org.apache.http.conn.ManagedHttpClientConnection;
import org.apache.http.conn.routing.HttpRoute;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.DefaultHttpResponseParserFactory;
import org.apache.http.impl.conn.ManagedHttpClientConnectionFactory;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.impl.io.DefaultHttpRequestWriterFactory;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.TrustStrategy;
import org.apache.http.util.EntityUtils;

import util.NoSNISSLSocketFactory;

public class TestSNI {

    public static void main(String[] args) {

         HttpEntity entity = null;

            try {

                final BasicCookieStore cookieStore = new BasicCookieStore();
                final HttpClientContext localContext = HttpClientContext.create();
                localContext.setCookieStore(cookieStore);

                // accept all certificates
                final SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
                    public boolean isTrusted(final X509Certificate[] arg0, final String arg1)
                            throws CertificateException {
                        return true;
                    }
                }).build();

                // set NoopHostnameVerifier()
                final Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder
                        .<ConnectionSocketFactory> create().register("http", new PlainConnectionSocketFactory())
                        .register("https", new SSLConnectionSocketFactory(
                                new NoSNISSLSocketFactory(sslContext.getSocketFactory()), new NoopHostnameVerifier()))
                        .build();

                final HttpConnectionFactory<HttpRoute, ManagedHttpClientConnection> connFactory = new ManagedHttpClientConnectionFactory(
                        new DefaultHttpRequestWriterFactory(),
                        new DefaultHttpResponseParserFactory());

                final PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(
                        socketFactoryRegistry, connFactory);

                final CloseableHttpClient httpclient = HttpClients.custom().setConnectionManager(cm).build();

                final HttpGet httpGet = new HttpGet("https://www.minervamedica.it");
                final CloseableHttpResponse rp = httpclient.execute(httpGet, localContext);

                entity = rp.getEntity();

                if (entity != null) {
                    System.out.println(EntityUtils.toString(entity));
                }

            } catch (final ClientProtocolException e) {
                System.out.println(e);
            } catch (final IOException e) {
                System.out.println(e);
            } catch (final Exception e) {
                System.out.println(e);
            } finally {
                EntityUtils.consumeQuietly(entity);
            }
    }

}

So it still seems there is no way to deal with the issue on Java > 8?

Edit 12.05.2019

Further debugging reveals that NoopHostnameVerifier, which overrides the method verify from javax.net.ssl.HostnameVerifier to return alwas true, is NOT called on Java 11. The exception already occurs in NoSNISSLSocketFactory, which extends SSLSocketFactory with an empty host:

    @Override
    public Socket createSocket(final Socket socket, final String host, final int port, final boolean autoClose)
            throws IOException {
        return sslSocketFactory.createSocket(socket, "", port, autoClose);
    }

So it appears that on Java 8 creating a Socket with an empty host did disable SNI, while on Java 11 this seems not to be the case?

Answer 1

Okay, I went through the source and found a way, although I'm not sure how enthusiastically I recommend it.

Although the javadoc for SSLParameters.setServerNames doesn't say so, if the value set is an empty List (with no elements), then ClientHandshaker actually sends no SNI at all. I suspect this is because the RFCs e.g. for 1.2 specify the min size as 1, prohibiting an empty list. (Compare to certificate_list in the Certificate message in TLS vs SSL; in SSL the min size was 1, and a client with no cert&key suitable for a server request didn't send the message at all, while in TLS it is 0, and a client with no suitable cert&key is explicitly specified to send a message containing an empty list.) While this is logical, since it is neither documented nor explicitly commented, I wouldn't be really happy relying on it.

Since it is pretty complicated (and fragile) to directly determine the other parameters needed, I think the best approach is to start from the existing parameters and modify, e.g. for SSLSocket:

SSLSocket s = SSLSocketFactory.getDefault() /* or other */ .createSocket("host", 443);
SSLParameters p = s.getSSLParameters();
p.setServerNames( new ArrayList<SNIServerName>() );
/* or j9+ p.setServerNames( List<SNIServerName>.of() ); */
s.setSSLParameters(p);
...

and for HttpsURLConnection, your original SSLSocketFactoryWrapper approach is quite close, except as above I would modify based on the actual parameters for the created SSLSocket and you must use the empty new ArrayList<SNIServerName>() and not .add anything to it.

Something very similar should work for Apache HttpClient, but I haven't gone through it, because I find that annoyingly like a maze of twisty little classes all alike.

PS: the source also confirms why varying sysprop jsse.enableSNIExtension won't work; that (like many others) is read and cached when JSSE is first loaded and not read subsequently. You could use reflection to break into the class and change the cached value, but let's not go there.