Kafka Connect and Kafka Broker version compatibili

Question

We have a "Kerberized Kafka cluster" running brokers version Apache Kafka 0.11.This cluster is managed by a different team and we dont have any control over this.

We are now trying to install Kafka Connect cluster on our own K8S cluster.

We were following this compatibility matrix https://docs.confluent.io/current/installation/versions-interoperability.html

According to this, we had to stick with Confluent Platform 3.3.3 images for Schema Registry and Kafka Connect pods since the Brokers version is 0.11

Also we want to enable Connect JMX metrics which was implemented here : https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A+Add+metrics+to+Kafka+Connect+framework

According to the documentation these JMX metrics are supported in Kafka 1.0.0

Questions are:

1) Can we use latest Confluent images (eg: version 5.2.0) with Kafka Brokers version 0.11?

2) With current setup (Confluent 3.3.3) I have enabled all the JMX metrics but we don't see any Kafka Connector based metrics. I see general Kafka Connect metrics, but not metrics per each Connector(eg. Source and Sink metrics).Is this because of the old CP version we use ?

Answer 1

Kafka Connect Workers that are included in Confluent Platform 3.2 and later are compatible with any Kafka broker that is included in Confluent Platform 3.0 and later

Note: Kafka Connect is not a Confluent Platform feature and is built over plain Apache Kafka API. Therefore, Connect instances using Kafka 1.0 and 2.0 client libraries still work with Kafka 0.11 brokers for basic communication protocols

If you're having Kerberos issues, those are likely unrelated

I see general Kafka Connect metrics, but not metrics per each Connector

In general, there is no such per connector metric such as "number of records read/written to a database", or at least, it would largely depend if the connectors actual implement their own metrics, but yes, the 3.3.x releases are all based on Kafka 0.11, not 1.0, where more Connect API metrics were added