Apache HttpClient 4.3 and x509 client certificate

Question

now I looking for solution regarding task how to rewrite deprecated solution for client side x509 certificate authentication via HttpComponentsMessageSender (not relevant).

For example, deprecated solution is:

    SSLSocketFactory lSchemeSocketFactory = new SSLSocketFactory(this.keyStore, this.keyStorePassword);
    Scheme sch = new Scheme("https", 443, lSchemeSocketFactory);

    DefaultHttpClient httpClient = (DefaultHttpClient)getHttpClient();
    httpClient.getConnectionManager().getSchemeRegistry().register(sch);

As new solution with CloseableHttpClient I am using:

    SSLContextBuilder sslContextBuilder = SSLContexts.custom()
            // this key store must contain the key/cert of the client
            .loadKeyMaterial(keyStore, keyStorePassword.toCharArray());

    if (trustStore != null) {
        // this key store must contain the certs needed and trusted to verify the servers cert
        sslContextBuilder.loadTrustMaterial(trustStore);
    }

    SSLContext sslContext = sslContextBuilder.build();

    LayeredConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext);

    // Create a registry of custom connection socket factories for supported
    // protocol schemes / https
    Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
            .register("https", sslsf)
            .register("http", new PlainConnectionSocketFactory())
            .build();

    PoolingHttpClientConnectionManager connPoolControl =
            new PoolingHttpClientConnectionManager(socketFactoryRegistry);
    setConnPoolControl(connPoolControl);
    getClientBuilder().setSSLSocketFactory(sslsf);

I still get 403 forbidden from server. But when I use "deprecated" version of the solution, it works great. SSL certificate is signed Thawte.

Any idea? Thanks

Answer 1

Below is the code for HttpClient 4.4+ (updated @Daniyar code for 4.4+)

import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.DefaultHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.ssl.SSLContexts;

public static CloseableHttpClient createApacheHttp4ClientWithClientCertAuth() {
    try {
        SSLContext sslContext = SSLContexts
                .custom()
                .loadTrustMaterial(null, new TrustSelfSignedStrategy())
                .build();

        SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslContext,
                new DefaultHostnameVerifier());

        Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory> create()
                .register("https", sslConnectionFactory)
                .register("http", new PlainConnectionSocketFactory())
                .build();

        HttpClientBuilder builder = HttpClientBuilder.create();
        builder.setSSLSocketFactory(sslConnectionFactory);
        builder.setConnectionManager(new PoolingHttpClientConnectionManager(registry));

        return builder.build();
    } catch (Exception ex) {

        return null;
    }
}

Answer 2

You need to create a keystore that containts the trusted CAs i.e. trust.jks. In this keystore you should put only the certificate of the server that your application is going to connect.

Then, you need a keystore for the identity of the server i.e. identity.jks. In this keystore you should store put the private key + certificate + CA chain under an alias (a name) that your application is going to use to authenticate itself with the server.

Then you could build the HttpClient like this:

public static HttpClient getHttpClient() throws KeyStoreException, CertificateException, NoSuchAlgorithmException, IOException, UnrecoverableKeyException, KeyManagementException {

    KeyStore identityKeyStore = KeyStore.getInstance("jks");
    identityKeyStore.load(DnieUtils.class.getClassLoader().getResourceAsStream("identity.jks"), "identity_password".toCharArray());

    KeyStore trustKeyStore = KeyStore.getInstance("jks");
    trustKeyStore.load(DnieUtils.class.getClassLoader().getResourceAsStream("trust.jks"), "trust_password".toCharArray());

    SSLContext sslContext = SSLContexts
            .custom()
            // load identity keystore
            .loadKeyMaterial(identityKeyStore, "identity_password".toCharArray(), new PrivateKeyStrategy() {
                @Override
                public String chooseAlias(Map<String, PrivateKeyDetails> aliases, Socket socket) {
                    return "identity_alias";
                }
            })
            // load trust keystore
            .loadTrustMaterial(trustKeyStore, null)
            .build();

    SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext,
            new String[]{"TLSv1.2", "TLSv1.1"},
            null,
            SSLConnectionSocketFactory.getDefaultHostnameVerifier());

    return HttpClients.custom()
            .setSSLSocketFactory(sslConnectionSocketFactory)
            .build();
}

To build the identity.jks, you need the CAs chain, the public key and the private key:

$1 = mycustomidentity

# make the keycert bundle for pkcs12 keystore
cat intermediate/certs/ca-chain.cert.pem \
    intermediate/certs/$1.cert.pem \
    intermediate/private/$1.key.pem \
    > intermediate/keycerts/$1.full-chain.keycert.pem

# generate the pkcs12 keystore with the alias of the server url
openssl pkcs12 -export \
    -in intermediate/keycerts/$1.full-chain.keycert.pem \
    -out intermediate/pkcs12s/$1.full-chain.p12 \
    -name $1 \
    -noiter -nomaciter

# .p12 to .jks
keytool -importkeystore -srckeystore $1.full-chain.p12 \
    -srcstoretype pkcs12 -srcalias $1 \
    -destkeystore identity.jks -deststoretype jks \
    -deststorepass identity_password -destalias identity_alias

For the trust.jks file you only need the certificate of the server (see https://stackoverflow.com/a/36427118/2692914 or https://stackoverflow.com/a/7886248/2692914), there is no problem in changing the alias:

# .crt, .cer into a .jks
keytool -import -alias trust_alias -file server_certificate.crt \
    -keystore trust.jks

Answer 3

Tomas, maybe it's too late, but I hope it will help others... There is the method, which I'm using to create CloseableHttpClient using Apache HttpClient 4.3:

public static CloseableHttpClient prepareClient() {
    try {           
        SSLContext sslContext = SSLContexts.custom().loadTrustMaterial(null, new TrustSelfSignedStrategy()).useTLS().build();
        HttpClientBuilder builder = HttpClientBuilder.create();
        SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslContext, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
        builder.setSSLSocketFactory(sslConnectionFactory);
        Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("https", sslConnectionFactory)
                .register("http", new PlainConnectionSocketFactory())
                .build();
        HttpClientConnectionManager ccm = new BasicHttpClientConnectionManager(registry);
        builder.setConnectionManager(ccm);
        return builder.build();
    } catch (Exception ex) {

        return null;
    }
}

Apache Foundation moved org.apache.http.conn.ssl.SSLContextBuilder, org.apache.http.conn.ssl.SSLContexts and org.apache.http.conn.ssl.SSLSocketFactory to deprecated starting with 4.4 version, There you can find Apache Client 4.5.2 API Depracated List. So, pervious method can be changed like this:

public static CloseableHttpClient prepareClient() {
    try {
        SSLContext sslContext = SSLContexts.custom()
                .loadTrustMaterial(null, new TrustSelfSignedStrategy()).build();
        HttpClientBuilder builder = HttpClientBuilder.create();
        SSLConnectionSocketFactory sslConnectionFactory = 
                new SSLConnectionSocketFactory(sslContext.getSocketFactory(), 
                        new NoopHostnameVerifier());
        builder.setSSLSocketFactory(sslConnectionFactory);
        Registry<ConnectionSocketFactory> registry = 
                RegistryBuilder.<ConnectionSocketFactory>create()
                .register("https", sslConnectionFactory)
                .register("http", new PlainConnectionSocketFactory())
                .build();
        HttpClientConnectionManager ccm = new BasicHttpClientConnectionManager(registry);
        builder.setConnectionManager(ccm);
        return builder.build();
    } catch (Exception ex) {
        LOG.error("couldn't create httpClient!! {}", ex.getMessage(), ex);
        return null;
    }
}

NoopHostnameVerifier

The NO_OP HostnameVerifier essentially turns hostname verification off. This implementation is a no-op, and never throws the SSLException.

If you need to verify hostname, you can use DefaultHostnameVerifier or you can implement your custom hostname verifier.