Apache kafka 2.0.0 version - Connection to node 1

Question

I'm using kafka version kafka_2.12-2.0.0 and received the below error after enabling SSL authentication. It seems to be working fine with previous versions: kafka_2.12-1.1.0, 2.11-0.10.2.2 etc.

I don't understand why it is not working with latest version 2.11-0.2.0.0? Has anyone observed the same issue that I'm facing right now with 2.0.0 version.

Below is my test environment docker config file.

listeners=PLAINTEXT://:9092,SSl://:9093
ssl.client.auth=required
ssl.keystore.location=/path/to/server.keystore
ssl.keystore.password=<Key store password>
ssl.key.password = <private key password>
ssl.truststore.location=/path/to/truststore.keystore
ssl.truststore.password=<trust store password>
security.inter.broker.protocol=SSL

And here's the error:

[2018-10-01 09:33:38,984] ERROR [Controller id=1, targetBrokerId=1] Connection to node 1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)

Can someone help me ?

Answer 1

Without more details it's hard to tell for sure, but 2.0.0 introduced a change of behaviour related to the handling of SSL connections.

As mentioned in the 2.0.0 upgrade notes, the broker setting ssl.endpoint.identification.algorithm is now set to https. This enforces hostname verification to prevent "man-in-the-middle" attacks.

To restore previous behaviour, you need to explicitely set this to an empty string.

ssl.endpoint.identification.algorithm=

Answer 2

Was also facing a similar issue. My issue, I was having Kafka server 1.1.1 running and was using Kafka client 2.1.0 to push records. Changing Kafka client to 1.1.1 solved my issue.

Hope this helps.