Can producer find the additions and removals of br

问题:

We knowthat, in kafka 0.7, we can specify zk.connect for producer, so producer can find the additions and removals of broker. But in kafka 0.8, we can't specify zk.connect for producer. Can producer in kafka 0.8 find that? If not, the scalability of the system is not worse than the 0.7 version?

回答1:

You can still use a ZooKeeper client to retrieve the broker list:

ZkClient zkClient = new ZkClient("localhost:2108", 4000, 6000, new BytesPushThroughSerializer());
List<String> brokerList = zkClient.getChildren("/brokers/ips");

According to that, you do not have to "hardcode" the broker list on client side and you are flexible as far as the system architecture is concerned. But anyway, this would add the ZooKeeper dependency again which is in fact an disadvantage for producer in several environments.

If you want to get a detailed view to the so called "cluster metadata API" solution, check out this link: https://issues.apache.org/jira/browse/KAFKA-369

Best

pre

P.S.: Sorry for reposting this to your other question - but the answer fits on both ;-)

回答2:

Little confused what exactly you are looking for, in 0.8 we must specify the list of brokers in the metadata.broker.list property

    Properties props = new Properties();
    props.put("metadata.broker.list", "broker1:9092,broker2:9092");

from the kafka consumer example they say

The property, “metadata.broker.list” defines where the Producer can find a one or more Brokers to determine the Leader for each topic. This does not need to be the full set of Brokers in your cluster but should include at least two in case the first Broker is not available. No need to worry about figuring out which Broker is the leader for the topic (and partition), the Producer knows how to connect to the Broker and ask for the meta data then connect to the correct Broker.

By saying additions do you mean adding new node to your cluster ?