Seeing “partition doesn't exist” warnings/fail

2019-05-01 09:53发布

问题:

I am using kafka 0.8.1.1. I have a 3 node kafka cluster with some topics having around 5 partitions. I planned to increase the number of nodes to 5 in cluster and moving some partitions from existing topics to the new brokers.

Previous partition state:
broker1 : topic1 { partition 0 }
broker2 : topic1 { partition 1,2}
broker3 : topic1 { partition 3,4}

New intended state:
broker1 : topic1 { partition 0}
broker2 : topic1 { partition 1}
broker3 : topic1 { partition 3}
broker4 : topic1 { partition 4}
broker5 : topic1 { partition 2}

command which I used:
bin/kafka-reassign-partitions.sh --reassignment-json-file partitions-to-move.json --zookeeper zkserver1:2181,zkserver2:2181,zkserver3:2181 --execute

partitions-to-move.json
{
    "partitions":[{"topic": "topic1","partition": 4,"replicas": [4] }],
    "version":1
}

Error messages that I am seeing:

[2015-12-22 11:19:46,447] WARN [KafkaApi-2] Produce request with correlation id 2462842 from client kafka-node-client on partition [topic1,4] failed due to Partition [topic1,4] doesn't exist on 3 (kafka.server.KafkaApis)

I did search but couldn't find any relevant answers. Appreciate any guidance/help to sort this out.