When using a custom X509KeyManager Java is not abl

Question

I'm working with Java7 and JAX-WS 2.2.

For a SOAP web service I need to create a custom X509KeyManager in order to find the correct certificate for each connecting client in a JKS keystore.

However, I'm already struggling to get the my custom key manager running. So far I'm using the default one (retrieved from the initialized KeyManagerFactory) and it basically works - but of course it doesn't select the correct certificate. So the first idea was to create a custom X509KeyManager which holds the original key manager, only writes out some log messages but generally uses the default behaviour.

For some reason that doesn't work at all. The SSL handshake cannot be established. After the ClientHello the log shows the following error:

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Thread-3, READ: TLSv1 Handshake, length = 149
*** ClientHello, TLSv1
RandomCookie:  GMT: 1476877930 bytes = { 207, 226, 8, 128, 40, 207, 47, 180, 146, 211, 157, 64, 239, 13, 201, 92, 158, 111, 108, 44, 223, 136, 193, 251, 33, 202, 7, 90 }
Session ID:  {}
Cipher Suites: [TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
***
%% Initialized:  [Session-3, SSL_NULL_WITH_NULL_NULL]
Thread-3, fatal error: 40: no cipher suites in common
javax.net.ssl.SSLHandshakeException: no cipher suites in common
%% Invalidated:  [Session-3, SSL_NULL_WITH_NULL_NULL]
Thread-3, SEND TLSv1 ALERT:  fatal, description = handshake_failure
Thread-3, WRITE: TLSv1 Alert, length = 2
Thread-3, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLHandshakeException: no cipher suites in common

I didn't remove any cipher suites at all to my knowledge! And the SSL handshake can be made with the same certificates.

This is my key manager:

public class CustomX509KeyManager extends X509ExtendedKeyManager
{
   private static final Logger LOG = Logger.getLogger( CustomX509KeyManager.class );
   private final X509KeyManager originalKeyManager;

   public CustomX509KeyManager(final X509KeyManager keyManager)
   {
      super();
      this.originalKeyManager = keyManager;
   }

   @Override
   public String chooseServerAlias(final String keyType, final Principal[] issuers,
     final Socket socket)
   {
      final String serverAliases=
            this.originalKeyManager.chooseServerAlias( keyType, issuers, socket );
      CustomX509KeyManager.LOG.info( "chooseServerAlias() " + serverAliases );
      return serverAliases;
   }

   ...
}

The other methods (not shown here) are just calling the corresponding methods in the originalKeyManager as well. During testing I never see the log message from the chooseServerAlias() method.

And it's initialized from another class in the getSslContext()method:

private KeyManager[] getKeyManagers(final KeyManagerFactory keyManagerFactory)
{
   final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();

   // replace any X509KeyManager with our own implementation
   for ( int i = 0; i < keyManagers.length; i++ )
   {
      if ( keyManagers[i] instanceof X509KeyManager )
      {
         keyManagers[i] =
               new CustomX509KeyManager( ( X509KeyManager ) keyManagers[i] );
      }
   }

   return keyManagers;
}

public SSLContext getSslContext()
{
  // create the KeyStore and load the JKS file
  final KeyStore keyStore = createKeyStore();

  // initialize key and trust manager factory
  final KeyManagerFactory keyManagerFactory =
        KeyManagerFactory.getInstance( KeyManagerFactory.getDefaultAlgorithm() );
  keyManagerFactory.init( keyStore, "changeit".toCharArray() );
  final TrustManagerFactory trustManagerFactory =
        TrustManagerFactory.getInstance( TrustManagerFactory.getDefaultAlgorithm() );
  trustManagerFactory.init( keyStore );

  // initialize the SSL context
  final SSLContext sslContext = SSLContext.getInstance( "TLS" );
  // sslContext.init( keyManagerFactory.getKeyManagers(),
  //      trustManagerFactory.getTrustManagers(), new SecureRandom() );
  sslContext.init( getKeyManagers( keyManagerFactory ),
        trustManagerFactory.getTrustManagers(), new SecureRandom() );
  return sslContext;
}

The commented lines show the original usage of the default key manager.

Any idea what's wrong? Why is the behaviour of using my CustomX509KeyManager so different than the default key manager that the handshake cannot be done? With the default key manager the encryption is negotiated for the TLS_DHE_RSA_WITH_AES_128_CBC_SHA algorithm which is available with the custom key manager as well but for some reason not chosen.

Update 1

I'm trying to connect with openssl in client mode to the server now but the server encounters the same problem using SSL. When I use the TLS protocol then the additional error message

Unsupported extension type_35, data:

appears.

Update 2

I can confirm that the above notice about unsupported extensions also appears upon successful handshakes so this is a false trace.

Answer 1

After a few days of trial & error I finally found my mistake!

In Java 7 a custom key manager should extend the X509ExtendedKeyManager which forces you to implement five methods of the interface X509KeyManager. However, there are two additional methods in the class X509ExtendedKeyManager which are not declared as abstract but must be overwritten for proper usage:

• chooseEngineClientAlias(String[], Principal[], SSLEngine)
• chooseEngineServerAlias(String, Principal[], SSLEngine)

After overwriting and implementing the methods by delegating the call to my originalKeyManager (which became of type X509ExtendedKeyManager as well) the SSL handshake finally succeeded.

Answer 2

It appears that you don't read keyfile anywhere in the code snippet. This is the reason of SSL_NULL_WITH_NULL_NULL. I suggest you implement X509KeyManager and read the file in constructor, so it can be used letter to select appropriate key. Something down this line (not all required methods depicted for the sake of short answer):

public class CustomX509KeyManager implements X509KeyManager
{
   private final KeyStore keyStore;
   private final String alias;
   private final char[] password;

   public CustomX509KeyManager(final String keyStoreFile, final char[] password, final String alias)
    throws IOException, GeneralSecurityException
   {
       this.alias = alias;
       this.password = password;
       synchronized(keyStoreFile)
       {
          InputStream stream = new FileInputStream(keyStoreFile);
          keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
          keyStore.load(stream, password);
          stream.close();
       }
   }

   @Override
   public PrivateKey getPrivateKey(String alias)
   {
       try {
           return (PrivateKey) keyStore.getKey(alias, password);
       } catch (Exception e) {
            e.printStackTrace();
            return null;
       }
    }

    @Override
    public X509Certificate[] getCertificateChain(String alias)
    {
        try {
            java.security.cert.Certificate[] certs = keyStore.getCertificateChain(alias);
            if (certs == null || certs.length == 0)
                return null;
            X509Certificate[] x509 = new X509Certificate[certs.length];
            for (int i = 0; i < certs.length; i++)
                x509[i] = (X509Certificate)certs[i];
            return x509;
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }          
    }

}

and then use it like

sslContext.init(new X509KeyManager[] { 
                    new CustomX509KeyManager(keyStoreFile, 
                        keyStorePass.toCharArray(), alias) }, null, null);