I am trying to access 6.x ES instance using High Level REST Client 6.7.2. Access to this ES instance is provided to me via hostname (https://****.azureedge.net), username & password.
My Spring Boot application is getting data from the same ES without issues when it runs from my dev environment (IDE), but throws SSLHandshakeException as soon as I try run it from Docker container (from my development machine or K8s cluster in cloud).
Container is made with base image: FROM debian:stretch-slim & OpenJDK 11.0.2 with Spring Boot necessary modules.
I made some progress debugging with -Djavax.net.debug=all. It turns out that while running in docker image just few first steps of usual SSL handshaking happen:
Produced ClientHello handshake message
WRITE: TLS13 handshake, length = 2352
Raw write
Raw read (0000: 15 03 03 00 02 02 28 ......( )
READ: TLSv1.2 alert, length = 2
Received alert message (
"Alert": {
"level" : "fatal",
"description": "handshake_failure"
}
)
followed by SSLHandshakeException:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:938)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1749)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1708)
at org.elasticsearch.client.SecurityClient.getSslCertificates(SecurityClient.java:508)
....
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Unknown Source)
at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source)
at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.decode(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
at java.base/javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:271)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316)
at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
When running from my local environment handshake looks uninterrupted:
Produced ClientHello handshake message
WRITE: TLS13 handshake, length = 460
Raw write
Raw read
READ: TLSv1.2 handshake, length = 155
Consuming ServerHello
ServerHello
Negotiated protocol version: TLSv1.3
Session initialized: Session(1560119025211|TLS_AES_256_GCM_SHA384)
WRITE: TLS13 change_cipher_spec, length = 1
Raw write
Raw read
READ: TLSv1.2 change_cipher_spec, length = 1
Consuming ChangeCipherSpec message
Raw read
READ: TLSv1.2 application_data, length = 27
...
Raw read
READ: TLSv1.2 application_data, length = 8469
Consuming server Certificate handshake message
... // here is the list of 3 certificates with "SHA256withRSA", "SHA256withRSA", "SHA1withRSA" signature algorithms
Found trusted certificate ⇢ SHA1withRSA
...
While running locally I noticed CN=Microsoft IT TLS CA 2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US, as well as CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE as issuers, maybe this is important, but I guess it is expected considering ES host address (Azure).
At the end I wanted to emphasise that I didn't need to do anything special to make this work in my macOS Java 11.0.2 development environment.
I already tried following, but that didn't change anything:
- Changing base Docker image from "slim" to non slim version
- Using OpenJDK 11.0.1 or 11.0.2
- Added cert from host to TrustStore that JVM is using in a runtime. (I checked in Docker container that there is indeed one more cert, but considering when handshake failure happens I imagine this is irrelevant)
- tried to enforce App with: "-Dcom.sun.net.ssl.enableECC=false", "-Djdk.tls.client.protocols=TLSv1.3", "-Dhttps.protocols=TLSv1.3", didn't help
Interesting: curl from Docker image with BasicAuth "talks" with same URL without issues (handshake completes) & small query returns results. I guess that curl and JVM are using different sources of trusted CAs inside the Docker, different algorithms for handshaking etc..
Thanks in advance for any help
