We received an email from AWS that basically says 'S3 is disabling SSLv3 Support, access will be cut-off in 15 days'. They then listed some buckets we have (one in production) that are 'currently accepting request from clients that specify SSLv3'. The full email is here, and other AWS users seem to have received one too:
https://gist.github.com/anonymous/4240c8af5208782c144c
My question is how do we test for this scenario, and what do we need to do to prepare for this cut-off date?
We use Rails 4.1 and the Fog (~> 1.28.0) and right_aws (~> 3.1.0) gems for AWS access and we're on Heroku. Our app provides signed HTTPS links to S3 resources to our browser users in our UI.
Is this just a client (browser) issue or something we need to understand better and test/fix?
fog uses excon for its http(s) transport. excon is a low-level pure-ruby http client, which relies on the ruby openssl bindings to work. Though it is possible to explicitly set an ssl version to use, excon doesn't, which to the best of my knowledge should mean that it negotiates with the server to choose what to use (so if the server asks for not SSLv3, it should cooperate).
I believe that should mean no action would be required here, but the specifics of all that vary a bit across Ruby and OpenSSL versions (not to mention that it is just a bit hard to introspect/understand the specifics of those bindings), so it is hard to say for certain. excon does support an ssl_version argument, which can be used to force a specific version if it does end up being a problem (this is just not a good general choice because it disallows negotiation and the specifics vary between ruby versions).
Hope that helps.
Deadline has been moved:
Based on the feedback received we are extending the deadline for discontinuing support of SSLv3 for securing connections to S3 buckets to 12:00 AM PDT May 20, 2015.
Update May 7 2015, 11:26 AM IST
In carrierwave initializer, put things as following,
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => Settings.carrier_wave.amazon_s3.access_key, # required
:aws_secret_access_key => Settings.carrier_wave.amazon_s3.secret_key, # required
:region => 'external-1' # optional, defaults to 'us-east-1'
}
config.fog_directory = Settings.carrier_wave.amazon_s3.bucket # required
#config.fog_host = 'http://aws.amazon.com/s3/' # optional, defaults to nil
config.fog_public = false # optional, defaults to true
config.fog_authenticated_url_expiration = 600
config.fog_attributes = {ssl_version: :TLSv1_2} #{'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
This worked for me, and have a look at the wireshark trace log.
1577 22.611358000 192.168.0.113 8.8.8.8 DNS 87 Standard query 0xffd8 A s3-external-1.amazonaws.com
1578 22.611398000 192.168.0.113 8.8.8.8 DNS 87 Standard query 0xbf2f AAAA s3-external-1.amazonaws.com
1580 22.731084000 8.8.8.8 192.168.0.113 DNS 103 Standard query response 0xffd8 A 54.231.1.234
1586 22.849595000 54.231.10.34 192.168.0.113 TLSv1.2 107 Encrypted Alert
1594 23.012866000 192.168.0.113 54.231.1.234 TLSv1.2 347 Client Hello
1607 23.310950000 192.168.0.113 54.231.1.234 TLSv1.2 204 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
1608 23.578966000 54.231.1.234 192.168.0.113 TLSv1.2 129 Change Cipher Spec, Encrypted Handshake Message
1609 23.579480000 192.168.0.113 54.231.1.234 TLSv1.2 427 Application Data
1610 23.868725000 54.231.1.234 192.168.0.113 TLSv1.2 299 Application Data
Update May 6 2015, 6-53 PM IST
Ok, After updating the Excon gem, we are able to see the TLSv1.2
protocol between our server and S3 servers.
bundle update excon
Wireshark trace log statements,
29 1.989230000 192.168.0.115 54.231.32.0 SSL 336 Client Hello
34 2.215461000 54.231.32.0 192.168.0.115 TLSv1.2 1494 Server Hello
40 2.219301000 54.231.32.0 192.168.0.115 TLSv1.2 471 Certificate
42 2.222127000 192.168.0.115 54.231.32.0 TLSv1.2 204 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
UPDATE May 6, 2015, 4-29 PM IST
After updating the hosts file, following is the wireshark trace log.
14 2.012094000 192.168.0.115 54.231.32.0 SSLv3 192 Client Hello
17 2.242423000 54.231.32.0 192.168.0.115 SSLv3 61 Alert (Level: Fatal, Description: Handshake Failure)
Please see the above wireshark request capture, when I upload a file from my local development rails on S3. As it shows, on initial handshake Amazon server uses SSLv3 and so my rails server sends all future requests with SSLv3.
Now, the question is, How can I change the bucket settings so that it would accept/initiate the process using TLS only? I have checked in amazon settings, there is nothing like that.
I have already changed my nginx to use TLS, but I think that is not needed because Rails will talk to S3 in the background using Excon as mentioned in above comment.
So, Please suggest what could be the best possible way to test this before 20th May, to make sure that it will not break on that day.
Any help would be great.
Just for information - My bucket name is like xyz.abc.com, so no - in the name.
AWS's official FAQ https://forums.aws.amazon.com/thread.jspa?threadID=179904&tstart=0
54.231.32.0 s3.amazonaws.com
54.231.32.1 <bucket name>.s3.amazonaws.com
54.231.32.3 <bucket name>.s3-external-1.amazonaws.com
Configure the above in your /etc/hosts
, replacing <bucket name>
with your bucket name.
NOTE: when using with a non us-east-1
bucket you may get redirect and failure responses. This has more to do with their adhoc infrastructure for testing this than anything else. So ignore that.
Create a "standard US bucket" and test with that instead. Remember to configure your app to use s3 region external-1
FWIW, my app using paperclip (4.2.0)
on ruby 2.1.4
works fine.
It is a client-side issue entirely , if the protocol that the client (e.g the browser) uses to issue requests over https is SSLv3 , than the ssl handshake will not succeed and these requests will fail. So it's the client that needs to disable SSLv3.
AWS's action is a follow-up on the POODLE vulnerability uncovered last year , and since then also all the AWS CloudFront distributions that use the *.cloudfront.net domain name have been updated with discontinued SSLv3 support .Now AWS is moving on to S3 to do the same.
I was able to force TLS using the following setting in my fog config:
connection_options: { ssl_version: :TLSv1_2 }
To test, update your host file (instructions from AWS):
54.231.32.0 s3.amazonaws.com
54.231.32.1 bucket.s3.amazonaws.com #replace bucket with your bucket name
54.231.32.3 bucket.s3-external-1.amazonaws.com #replace bucket with your bucket name
I was able to connect successfully. Also, If you change the setting to :SSLv3 you'll get an error. Good Luck!