Mutual authentication with Tomcat 7

Question

I'm trying to set up a Java web service running in Tomcat 7 to use mutual (2-way) authentication. It seems like no matter what I do, connecting to the service on the secure port isn't working.

Here's what I did to create certificates and keystores and such:

//create the key and certificate for the tomcat server.
keytool -genkey -v -alias tomcat -keyalg RSA -validity 3650 -keystore tomcat.keystore

//create the key and certificate for the client machine.
keytool -genkey -v -alias clientkey -keyalg RSA -storetype PKCS12 -keystore client.p12

//export the client key
keytool -export -alias clientkey -keystore client.p12 -storetype PKCS12 -rfc -file client.cer

//import the client key into the server keystore
keytool -import -v -file client.cer -keystore tomcat.keystore

Here's the connector in the server.xml file:

<Connector port="8443"
    maxThreads="150"
    scheme="https"
    secure="true"
    sslProtocol="TLS"
    clientAuth="true"
    keystoreFile="tomcat.keystore"
    keystorePass="tomcat"
    truststoreFile="tomcat.keystore"
    truststorePass="tomcat"/>

The tomcat-users.xml file looks like this:

<tomcat-users>
    <role rolename="tomcat"/>
    <role rolename="admin"/>
    <!-- note that the actual values for CN, OU, O, L, ST are different, but they match the values created in the client certificate -->
    <user username="CN=name, OU=unit, O=org, L=locality, ST=state, C=US" password="null" roles="admin" />
</tomcat-users>

The following are set on startup:

-Djavax.net.ssl.keyStoreType=jks
-Djavax.net.ssl.keyStore=tomcat.keystore
-Djavax.net.ssl.keyStorePassword=tomcat
-Djavax.net.ssl.trustStore=tomcat.keystore
-Djavax.net.ssl.trustStorePassword=tomcat
-Djavax.net.debug=SSL

Finally, I copied the client.p12 file to my client machine, and imported it into Firefox's client certificates.

First problem: When I hit an endpoint on my service (example - https://my.server.com:8443/test) from Firefox, I get the response "Secure Connection Failed". SSL received a record that exceeded the maximum permissible length. (Error code: ssl_error_rx_record_too_long)

Second problem: I don't really want to run this connector on port 8443. I want to run it on port 7800 (which is our company standard for HTTPS). When I change the port on the Connector to 7800 and try to hit the endpoint (example - https://my.server.com:7800/test) then it never resolves the page.

So, somewhere I'm obviously missing a crucial piece. Can anyone see my error?

UPDATE: after feedback from @Dave G

Running the command:

openssl s_client -connect localhost:8443 -showcerts

produces the following output:

CONNECTED(00000003)
140642290976584:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:766:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 263 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

I also added -Djavax.net.debug=SSL to the startup. This generates the following in the beginning of the catalina.out file:

trustStore is: tomcat.keystore
trustStore type is : jks
trustStore provider is :
init truststore
adding as trusted cert:
  Subject: CN=localhost, OU=unit, O=org, L=Springfield, ST=MO, C=US
  Issuer:  CN=localhost, OU=unit, O=org, L=Springfield, ST=MO, C=US
  Algorithm: RSA; Serial number: 0x5485b5a5
  Valid from Mon Dec 08 14:28:53 UTC 2014 until Thu Dec 05 14:28:53 UTC 2024

adding as trusted cert:
  Subject: CN=William Jackson, OU=unit, O=org, L=Springfield, ST=MO, C=US
  Issuer:  CN=William Jackson, OU=unit, O=org, L=Springfield, ST=MO, C=US
  Algorithm: RSA; Serial number: 0x5485b6af
  Valid from Mon Dec 08 14:33:19 UTC 2014 until Sun Mar 08 14:33:19 UTC 2015

trigger seeding of SecureRandom
done seeding SecureRandom

And then a LOT of:

Ignoring unavailable cipher suite: <suite name>
Ignoring unsupported cipher suite: <suite name>

Answer 1

It took me some time to get it working correctly using Openssl certificates, drafting my notes so that it may help others visiting this page.

Step 1: Create your own root CA

~/openssl$ mkdir -m 0700 /home/ubuntu/openssl/CA /home/ubuntu/openssl/CA/certs /home/ubuntu/openssl/CA/crl /home/ubuntu/openssl/CA/newcerts /home/ubuntu/openssl/CA/private
~/openssl$ touch /home/ubuntu/openssl/CA/indext.txt
~/openssl$ echo 1000 >> /home/ubuntu/openssl/CA/serial
~/openssl$ mv karun-tomcat-root-ca.key CA/private/

~/openssl$ sudo vi /etc/openssl.cnf
    # Make changes here
    dir = /home/ubuntu/openssl/CA
    #optionally change policy definitions as well
~/openssl$ openssl genrsa -des3 -out karun-tomcat-root-ca.key 2048

  #In below command make sure to use CN=<hostname of your machine>
~/openssl$ openssl req -new -x509 -days 36520 -key karun-tomcat-root-ca.key -out karun-tomcat-root-ca.crt -config openssl.cnf

~$ sudo cp ~/openssl/CA/certs/karun-tomcat-root-ca.crt /usr/share/ca-certificates/

  # make sure in the UI you enable/select the certificate created above
~$ sudo dpkg-reconfigure ca-certificates
  # Now reboot ubuntu machine just to make sure certificates are loaded successfully and tomcat picks it

Step 2: Create Tomcat Server's Key Pair

~$ openssl genrsa -out tomcat-server.key 2048

   # Use common name = <Give IP address>, department = Tomcat Server CSR
~$ openssl req -new -sha256 -config ~/openssl/openssl.cnf -key tomcat-server.key -out tomcat-server.csr
~$ openssl x509 -req -sha256 -days 36520 -in tomcat-server.csr -signkey tomcat-server.key -CA ~/openssl/CA/certs/karun-tomcat-root-ca.crt -CAkey ~/openssl/CA/private/karun-tomcat-root-ca.key -CAcreateserial -out tomcat-server.crt 
~$ openssl pkcs12 -export -name karun-tomcat-server-cert -in tomcat-server.crt -out tomcat-server.p12 -inkey tomcat-server.key -CAfile ~/openssl/CA/certs/karun-tomcat-root-ca.crt -caname karun-root -chain

~$ keytool -importkeystore -destkeystore tomcat-server.jks -srckeystore tomcat-server.p12 -srcstoretype pkcs12 -alias karun-tomcat-server-cert

~$ keytool -import -alias karun-root -keystore tomcat-server.jks -trustcacerts -file ~/openssl/CA/certs/karun-tomcat-root-ca.crt

# **(LATER)** Run this once client cert is generated
~$ keytool -importkeystore -alias karun-tomcat-client-cert -srckeystore ~/client-certs/tomcat-client.p12 -srcstoretype PKCS12 -destkeystore tomcat-server.jks -deststoretype JKS

# **(LATER)** Run this once tomcat server started successfully
~$ openssl s_client -connect localhost:8443 -cert ~/client-certs/tomcat-client.crt -key ~/client-certs/tomcat-client.key -debug -showcerts 

Step 3: Create Client Side Key Pair

~$ openssl genrsa -out tomcat-client.key 2048
  # Use common name = <tomcat-user.xml's user say 'admin'>, department = Tomcat Client CSR
~$ openssl req -new -sha256 -config ~/openssl/openssl.cnf -key tomcat-client.key -out tomcat-client.csr
~$ openssl x509 -req -sha256 -days 36520 -in tomcat-client.csr -signkey tomcat-client.key -CA ~/openssl/CA/certs/karun-tomcat-root-ca.crt -CAkey ~/openssl/CA/private/karun-tomcat-root-ca.key -CAcreateserial -out tomcat-client.crt 
~$ openssl pkcs12 -export -name karun-tomcat-client-cert -in tomcat-client.crt -out tomcat-client.p12 -inkey tomcat-client.key -CAfile ~/openssl/CA/certs/karun-tomcat-root-ca.crt -caname karun-root -chain
~$ (optional step) keytool -importkeystore -destkeystore tomcat-client.jks -srckeystore tomcat-client.p12 -srcstoretype pkcs12 -alias karun-tomcat-client-cert
~$ (optional step) keytool -import -alias root -keystore tomcat-client.jks -trustcacerts -file ~/openssl/CA/certs/karun-tomcat-root-ca.crt

Step 4: Tomcat Changes

# Make this change in server.xml of tomcat server
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
           maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
           keystoreFile="/opt/tomcat/openssl-certs/tomcat-server.jks"
           keystorePass="password"
           keyAlias="karun-tomcat-server-cert"
           truststoreFile="/opt/tomcat/openssl-certs/tomcat-server.jks"
           truststorePass="password"
           clientAuth="true" sslProtocol="TLS" />

Step 5: Restart Tomcat Server && check logs to ensure no errors at bootup

Step 6: Upload Client cert to browser

In your browser, eg: firefox, navigate Preferences -> Advanced -> Certificate -> View Certificates -> Your Certificates

Import "tomcat-client.p12"

https://<tomcat ip>:8443/

References

http://pages.cs.wisc.edu/~zmiller/ca-howto/

http://www.area536.com/projects/be-your-own-certificate-authority-with-openssl/

Answer 2

Ok - after digging a lot more, I finally got this working. Much thanks to @Dave G and this tutorial: Configuring two-way SSL authentication on Tomcat from which most of these instructions are paraphrased.

Generally, the steps to get mutual authentication functional are as follows:

1. Create a certificate for the tomcat server. The client has to trust this certificate.
2. Create a keystore for the tomcat server, and import the server certificate into it.
3. Create a certificate for the client. The server has to trust this certificate.
4. Import the client certificate into the server keystore
5. Update the tomcat server.xml file with the correct Connector XML.

The above steps are necessary on the server. Once completed, to set up the client, do the following:

1. Copy the client certificate from the server to the client.
2. Use the client certificate when communicating with the server (this process varies with the nature of the client application).

For the certificate configuration, I executed the following on the server machine:

# For the following commands, set the values in parenthesis to be whatever makes sense for your environment.  The parenthesis are not necessary for the command.

# This is an all-in-one command that generates a certificate for the server and places it in a keystore file, while setting both the certifcate password and the keystore password.
# The net result is a file called "tomcat.keystore". 

keytool -genkeypair -alias (serveralias) -keyalg RSA -dname "CN=(server-fqdn),OU=(organizationalunit),O=(organization),L=(locality),ST=(state),C=(country)" -keystore tomcat.keystore -keypass (password) -storepass (password)

# This is the all-in-one command that generates the certificate for the client and places it in a keystore file, while setting both the certificate password and the keystore password.
# The net result is a file called "client.keystore"

keytool -genkeypair -alias (clientalias) -keyalg RSA -dname "CN=(client),OU=(organizationalunit),O=(organization),L=(locality),ST=(state),C=(country)" -keypass (password) -keystore client.keystore -storepass (password) 

# This command exports the client certificate.  
# The net result is a file called "client.cer" in your home directory.

keytool -exportcert -rfc -alias (clientalias) -file client.cer -keypass (password) -keystore client.keystore -storepass (password)

# This command imports the client certificate into the "tomcat.keystore" file.

keytool -importcert -alias (clientalias) -file client.cer -keystore tomcat.keystore -storepass (password) -noprompt

Certificates should now be set up appropriately. The next step is to configure your connector in the tomcat server.xml. Add a connector element that looks like this:

<Connector port="8443"
    maxThreads="150"
    scheme="https"
    secure="true"
    SSLEnabled="true"
    truststoreFile="/full/path/to/tomcat.keystore"
    truststorePass="(password)"
    keystoreFile="/full/path/to/tomcat.keystore"
    keystorePass="(password)"
    clientAuth="true"
    keyAlias="serverkey"
    sslProtocol="TLS"/>      

Note that in the above XML:

1. The "port" attribute can be whatever you want.
2. The "keystoreFile" and "truststoreFile" attributes should be full paths. Tomcat does not look in the same directory as server.xml by default.
3. The "keystorePass" and "truststorePass" attributes should match the (password) value you used in the creation of the tomcat.keystore file.
4. The "clientAuth" attribute must be set to "true". This is what triggers mutual authentication.

Additionally, in the server.xml, ensure that you DO NOT have an AprLifecycleListner defined. The XML for that listener will look something like this:

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />

That element should be delete/commented out. The AprLifecycleListener does not get configured the same way as described above, and will not work with these instructions.

Restart tomcat. The server configuration should be complete.

I tested my work using Firefox, because it's easy to add client certificates to it. Open up Firefox and try to connect to an endpoint of your tomcat service on the port defined in your connector.

Ex: https://mytomcatdomain.com:8443/test

When you do this, you should get the standard alert from Firefox about an untrusted connection because we created a self-signed certificate for our Tomcat server. Add an exception for the certificate so that our client (Firefox) trusts our server (Tomcat).

Once you've added the exception, you should get a "Secure Connection Failed" message. The error code is "ssl_error_bad_cert_alert". This confirms that our Tomcat server is requesting authentication from the client. The request is failing because we have not configured Firefox to send our trusted client certificate yet.

To configure Firefox, we need to do a little more magic:

// Create a file called DumpPrivateKey.java.  The contents should look like so:
public class DumpPrivateKey {
public static void main(String[] args) throws Exception {
  final String keystoreName = args[0];
    final String keystorePassword = args[1];
    final String alias = args[2];
    java.security.KeyStore ks = java.security.KeyStore.getInstance("jks");
    ks.load(new java.io.FileInputStream(keystoreName), keystorePassword.toCharArray());
    System.out.println("-----BEGIN PRIVATE KEY-----");
    System.out.println(new sun.misc.BASE64Encoder().encode(ks.getKey(alias, keystorePassword.toCharArray()).getEncoded()));
    System.out.println("-----END PRIVATE KEY-----");
  }
}

Compile the java file with the following command:

javac DumpPrivateKey.java

Now we're going to use this little utility to extract a key from the client.keystore file we create above. Copy the client.keystore and client.cer files into the same directory as your DumpPrivateKey class. Execute the following:

# This extracts the client key from the client keystore

java DumpPrivateKey client.keystore (password) clientkey > clientkey.pkcs8

# This creates a client.p12 file that can be used by Firefox

openssl pkcs12 -export -in client.cer -inkey clientkey.pkcs8 -password pass:(password) -out client.p12

Note that in the above code, (password) should be the password you used to create the client.keystore.

Open up Firefox preferences. Click on the "Certificates" tab. Click on the "View Certificates" button. Click on the "Your Certificates" tab.

Click on the "Import" button and browse to the "client.p12" file that was created previously. You should be prompted to enter the password for the client certificate.

Assuming the "client.p12" was imported successfully, you can now refresh you Firefox page, and you should get a successful response from your Tomcat server endpoint.

Answer 3

I would try the following steps

1. Spool up container as you have configured on 8443.
2. Run up your client application with -Djavax.net.debug=SSL

That command will spool out PILES of information. What you need to check on that is that the server is presenting a list of CA's that it will accept for mutual authentication. If the listed CAs do not contain your certificate then the client will have no idea how to locate a match for the server.

This can be made much easier using the openssl command 's_client'

openssl s_client -connect localhost:8443 -showcerts

That will format out some information that can be incalculable in their value of debugging this.

If the server does not present a list of "acceptable" CAs you will have to do some magic when you produce your certificate set.

Let me know what you find out and I can hopefully steer you in the right direction.

OP added additional information

Ok so the following is a bit of a problem for you:

---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 263 bytes
---

Two things jump out immediately

1. The server doesn't have a peer certificate
2. There are no client CA names listed

So for (1):

1. make sure your keystore in fact does have the alias 'tomcat' in it using keytool.
2. the store/key password stuff in tomcat is goofy. for sanity's sake, add both keystorePassword and keyPassword attributes to your connector with identical values. The documentation for Tomcat 7 indicates that the keystorePass will default to the keyPass if not set. If your keyPass and keystorePass are the same set the keyPass attribute only.

Now for (2) we really need to have (1) working first - so get that up and running and we'll see where we are at that point.

Answer 4

@wbj, export of PrivateKeyEntry from JKS to PKCS #12 can be done much more easier:

keytool -importkeystore -srckeystore client.keystore -destkeystore client.p12 -deststoretype PKCS12 -srcalias client -deststorepass <password> -destkeypass <password>

Cheers.