What happens when I partition data by key and then later on add a new partition to the topic in Kafka?
Will there be any change to the existing record? And how will the future data be partitioned?
What happens when I partition data by key and then later on add a new partition to the topic in Kafka?
Will there be any change to the existing record? And how will the future data be partitioned?
Partitioning of existing data doesn't change when new partitions are added to a particular topic. Kafka will not attempt to re-distribute existing records and this modification will only have effect on new records. Note that by default, Kafka partitions data using hash(key) % noOfPartitions
in order to ensure that records with the same key are added to the same partition. Data with different key will be added to partitions in a round-robin fashion.