difference between groupid and consumerid in Kafka

Question

I am new to Kafka. I noticed in Consumer configuration that has two ids. one is group.id ( mandatory ) and second one is consumer.id ( non Mandatory ). Please tell why 2 Ids and difference.

Answer 1

Consumers groups is a Kafka abstraction that enables supporting both point-to-point and publish/subscribe messaging. A consumer can join a consumer group (let us say group_1) by setting its group.id to group_1. Consumer groups is also a way of supporting parallel consumption of the data i.e. different consumers of the same consumer group consume data in parallel from different partitions.

In addition to group.id, each consumer also identifies itself to the Kafka broker using consumer.id. This is used by Kafka to identify the currently ACTIVE consumers of a particular consumer group.

Read this documentation for more details.