Within the remote kafka cloud cluster, the kafka brokers will be updated to a new version (5.1) and therefore a new kafka protocol applies.
Now I should update my kafka client to being able to connect. Right now I am using following kafka relevant dependencies in my spring-boot application:
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-parent</artifactId>
<version>1.3.8.RELEASE</version>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-kafka</artifactId>
<version>2.0.1.RELEASE</version>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<version>1.0.3.RELEASE</version>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>1.0.3.RELEASE</version>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.9.0.1</version>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.9.0.1</version>
I already searched for an appropriate compatibility matrix for kafka-clients and kafka-brokers.
- Does anyone have an idea how to upgrade successfully ?
- Does anyone have a comptability matrix for this dependencies?
There's a link to the confluent matrix on the Spring for Apache Kafka project page (along with spring-kafka/kafka-clients compatibility).
0.9 is very, very old.
Typically, clients/brokers newer than 0.10.2.0 can talk to each other, but if records have headers, you will need a client >= 0.11.0.0.
From https://github.com/spring-cloud/spring-cloud-stream/wiki/Kafka-Client-Compatibility
Compatibility matrix
+--------+--------------+------------------+---------------+-------------+
| Spring | Spring for | Spring | kafka-clients | Kafka |
| Cloud | Apache Kafka | Integration | | Broker |
| Stream | | for Apache Kafka | | |
+--------+--------------+------------------+---------------+-------------+
| 2.1.x | 2.2.x | 3.1.x | 2.0.0, | 2.0.0, |
| | | | 1.1.x, | 1.1.x, |
| | | | 1.0.x(*) | 1.0.x, |
| | | | | 0.11.0.x(*) |
+--------+--------------+------------------+---------------+-------------+
| 2.0.x | 2.1.x | 3.0.x | 1.1.x, | 1.1.x, |
| | | | 1.0.x(*) | 1.0.x, |
| | | | | 0.11.0.x(*) |
+--------+--------------+------------------+---------------+-------------+
| 1.3.x | 1.3.x, | 2.3.x, | 0.11.0.x(**), | 0.10.x.x |
| | 1.2.x, | 2.2.x, | 0.10.2.x | or higher |
| | 1.1.x | 2.1.x | | |
+--------+--------------+------------------+---------------+-------------+
| 1.2.x | 1.2.x, | 2.2.x, | 0.10.1.x | 0.10.x.x |
| | 1.1.x | 2.1.x | | or higher |
+--------+--------------+------------------+---------------+-------------+
Other sources
- https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix
- https://spring.io/projects/spring-kafka
