Unable to configure SSL for Kafka Connect REST API

Question

I'm trying to configure SSL for Kafka Connect REST API (2.11-2.1.0).

The problem

I tried two configurations (worker config):

• with listeners.https. prefix

listeners=https://localhost:9000
listeners.https.ssl.keystore.location=/mypath/keystore.jks
listeners.https.ssl.keystore.password=mypassword
listeners.https.ssl.key.password=mypassword

• and without listeners.https. prefix

listeners=https://localhost:9000
ssl.keystore.location=/mypath/keystore.jks
ssl.keystore.password=mypassword
ssl.key.password=mypassword

Both configurations starts OK, and show following exception when trying to connect to https://localhost:9000 :

javax.net.ssl.SSLHandshakeException: no cipher suites in common

In log, I see that SslContextFactory was created with any keystore, but with ciphers:

210824 ssl.SslContextFactory:350 DEBUG: Selected Protocols [TLSv1.2, TLSv1.1, TLSv1] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
210824 ssl.SslContextFactory:351 DEBUG: Selected Ciphers   [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, ...]
210824 component.AbstractLifeCycle:177 DEBUG: STARTED @10431ms SslContextFactory@42f8285e[provider=null,keyStore=null,trustStore=null]

What I did

As I know that password from keystore is absolutely correct, I digged into source code, and started to debug.

Finally, I find out that neither plain ssl.* nor prefixed listeners.https.ssl.* configurations are not taken into account, and it turns that there is not possibility to configure SSL for Kafka Connect REST API currently.

Call sequence is:

1. RestServer.createConnector
2. SSLUtils.createSslContextFactory
3. AbstractConfig.valuesWithPrefixAllOrNothing

Last method is the reason of troubles.

If we have listeners.https. properties, they cannot be returned, because they filtered out at line 254 (since WorkerConfig contains no properties with the prefix).

Otherwise, if we have unprefixed ssl. properties, they also not returned, because values field contains only known properties from the same WorkerConfig (values are result of ConfigDef.parse).

Am I missing something, and has anyone successfully configured SSL for kafka connect rest api ?

Answer 1

I haven't test Connect REST API, but KafkaTemplate send and recieves messages with ssl. From your configuration i may assume two problems:

• you not specified the truststore (for certificate chain check)
• you used absolute path, but spring keystore-location interprets as
  relative to /webapp

I tried test application from examples: https://memorynotfound.com/spring-kafka-and-spring-boot-configuration-example/ and https://gist.github.com/itzg/e3ebfd7aec220bf0522e23a65b1296c8

Tested with springboot 2.0.4.RELEASE, used kafka library

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

and this my application.properties content:

spring.application.name=my-stream-app
spring.kafka.bootstrap-servers=localhost:9093
spring.kafka.ssl.truststore-location=kafka.server.truststore.jks
spring.kafka.ssl.truststore-password=123456 
spring.kafka.ssl.keystore-location=kafka.server.keystore.jks
spring.kafka.ssl.keystore-password=123456 
spring.kafka.ssl.key-password=123456
spring.kafka.properties.security.protocol=SSL

spring.kafka.consumer.group-id=properties test-consumer-group        
app.topic.foo=test 

fragment of kafka server configuration:

listeners=SSL://localhost:9093

ssl.truststore.location=/home/legioner/kafka.server.truststore.jks
ssl.truststore.password=123456
ssl.keystore.location=/home/legioner/kafka.server.keystore.jks
ssl.keystore.password=123456
ssl.key.password=123456

Answer 2

Try export KAFKA_OPTS=-Djava.security.auth.login.config=/apps/kafka/conf/kafka/kf_jaas.conf where kf_jaas.conf contains ZooKeeper client authentication