Java 11 SSL exception: unable to find valid certif

Question

During an attempt to upgrade one of my applications from java 8 to java 11, I run into a weird SSL exception about "unable to find valid certification"

the provisioning of this application overwrites default jdk / jre cacerts with cacerts from ca-certificates-java package so before and after the upgrade, the certificates in the key store should be the same (this is also verified using keytools):

# java 8
ls -l /usr/lib/jvm/java-1.8.0/jre/lib/security/cacerts
lrwxrwxrwx 1 root root 41 Sep 11 22:52 /usr/lib/jvm/java-1.8.0/jre/lib/security/cacerts -> /etc/pki/java/cacerts

# java 11
ls -l /usr/lib/jvm/jdk-11.0.2/lib/security/cacerts
lrwxrwxrwx 1 root root 21 Oct 29 21:22 /usr/lib/jvm/jdk-11.0.2/lib/security/cacerts -> /etc/pki/java/cacerts

Investigated a few days but I didn't get myself anywhere. I think this may have been caused by changes in SunJSSE security Provider between version 1.8 and 11.0 but I cannot confirm.

I'm not familiar with java security and would like to reach out for help. The example has been simplified to this project and the instructions on reproducing this issue is in the README.md

Exception call stack:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:641)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:460)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
    at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
    at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1048)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:995)
    at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1502)
    at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1516)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1400)
    at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441)
    ... 17 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
    at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:290)
    at java.base/sun.security.validator.Validator.validate(Validator.java:264)
    at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
    at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:619)
    ... 32 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
    at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
    ... 38 more

Note: the project was set up with netty and the ssl context was set up using netty's SslContext. I've tried using javax.net.ssl.SSLContext too:

SSLEngine engine = SSLContext.getDefault().createSSLEngine();
engine.setUseClientMode(true);

And this results in the same exception and call stack.

Answer 1

I ran your docker image with Java debugging (-Djavax.net.debug=all) and found the following. This Github Issue may help you find a solution.

javax.net.ssl|DEBUG|0F|nioEventLoopGroup-2-1|2019-11-06 06:13:01.316 UTC|CertificateMessage.java:357|Consuming server Certificate handshake message (
"Certificates": [
  "certificate" : {
    "version"            : "v3",
    "serial number"      : "00 90 76 89 18 E9 33 93 A0",
    "signature algorithm": "SHA256withRSA",
    "issuer"             : "CN=invalid2.invalid, OU="No SNI provided; please fix your client."",
    "not before"         : "2015-01-01 24:00:00.000 UTC",
    "not  after"         : "2030-01-01 24:00:00.000 UTC",
    "subject"            : "CN=invalid2.invalid, OU="No SNI provided; please fix your client."",
    "subject public key" : "RSA",
    "extensions"         : [
      {

Answer 2

I had similar issue while we migrated to JAVA11 from JAVA8, if I remember, as a workaround copying the contents of /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.172-9.b11.fc28.x86_64/jre/lib/security/cacerts to $JAVA_HOME/jre/lib/security/cacerts solved the problem.