SSL problems with S3/AWS using the Java API: “host

Question

Amazon "upgraded" the SSL security in its AWS Java SDK in the 1.3.21 version. This broke access any S3 buckets that have periods in their name when using Amazon's AWS Java API. I'm using version 1.3.21.1 which is current up to Oct/5/2012. I've provided some solutions in my answer below but I'm looking for additional work arounds to this issue.

If you are getting this error, you will see something like the following message in your exceptions/logs. In this example, the bucket name is foo.example.com.

INFO: Unable to execute HTTP request: hostname in certificate didn't match:
       <foo.example.com.s3.amazonaws.com> != <*.s3.amazonaws.com>
       OR <*.s3.amazonaws.com> OR <s3.amazonaws.com>
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:220)
at org.apache.http.conn.ssl.StrictHostnameVerifier.verify(StrictHostnameVerifier.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:149)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:130)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)

You can see documentation of this problem on the AWS S3 discussion forum:

https://forums.aws.amazon.com/thread.jspa?messageID=387508&#387508

Amazon's response to the problem is the following.

We should be able to fix this by using the older path style method of bucket addressing (instead of the newer virtual host style addressing) for buckets with this naming pattern. We'll get started on the fix and ensure that our internal integration tests have test cases for buckets names containing periods.

Any workaround or other solutions? Thanks for any feedback.

Answer 1

Amazon released version 1.3.22 which resolves this issue. I've verified that our code now works. To quote from their release notes:

Buckets whose name contains periods can now be correctly addressed again over HTTPS.

There are a couple of solutions that I can see, aside from waiting till Amazon releases a new API.

1. Obviously you could roll back to 1.3.20 version of the AWS Java SDK. Unfortunately I needed some of the features in 1.3.21.

2. You can replace the org.apache.http.conn.ssl.StrictHostnameVerifier in the classpath. This is a hack however which will remove all SSL checking for Apache http connections I think. Here's the code that worked for me: http://pastebin.com/bvFELdJE

3. I ended up downloading and building my own package from the AWS source jar. I applied the following approximate patch to the HttpClientFactory source.

   ===================================================================
   --- src/main/java/com/amazonaws/http/HttpClientFactory.java     (thirdparty/aws)      (revision 20105)
   +++ src/main/java/com/amazonaws/http/HttpClientFactory.java     (thirdparty/aws)    (working copy)
   @@ -93,7 +93,7 @@
   
                           SSLSocketFactory sf = new SSLSocketFactory(
                                   SSLContext.getDefault(),
   -                               SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
   +                               SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
   

4. The right fix is to change from domain-name bucket handling to path based handling.

Btw, the following seems like it might work but it does not. The AWS client specifically requests the STRICT verifier and does not use the default one:

SSLSocketFactory.getSystemSocketFactory().setHostnameVerifier(
    SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

Answer 2

Turns out that Amazon "upgraded" the SSL security on S3 in late September 2012. This broke access any S3 buckets that have periods in their name when using Amazon's AWS Java API.

This is inaccurate. S3's SSL wildcard matching has been the same as when S3 launched back in 2006. What's more likely is that the AWS Java SDK team enabled stricter validation of SSL certificates (good), but ended up breaking bucket names that have been running afoul of S3's SSL cert (bad).

The right answer is that you need to use path-style addressing instead of DNS-style addressing. That is the only secure way of working around the issue with the wildcard matching on the SSL certificate. Disabling the verification opens you up to Man-In-The-Middle attacks.

What I don't presently know is if the Java SDK provides this as a configurable option. If so, that's your answer. Otherwise, it sounds like the Java SDK team said "we'll add this feature, and then add integration tests to make sure it all works."