Can producer find the additions and removals of br

2019-04-15 23:37发布

We knowthat, in kafka 0.7, we can specify zk.connect for producer, so producer can find the additions and removals of broker. But in kafka 0.8, we can't specify zk.connect for producer. Can producer in kafka 0.8 find that? If not, the scalability of the system is not worse than the 0.7 version?

2条回答
可以哭但决不认输i
2楼-- · 2019-04-15 23:56

You can still use a ZooKeeper client to retrieve the broker list:

ZkClient zkClient = new ZkClient("localhost:2108", 4000, 6000, new BytesPushThroughSerializer());
List<String> brokerList = zkClient.getChildren("/brokers/ips");

According to that, you do not have to "hardcode" the broker list on client side and you are flexible as far as the system architecture is concerned. But anyway, this would add the ZooKeeper dependency again which is in fact an disadvantage for producer in several environments.

If you want to get a detailed view to the so called "cluster metadata API" solution, check out this link: https://issues.apache.org/jira/browse/KAFKA-369

Best

pre

P.S.: Sorry for reposting this to your other question - but the answer fits on both ;-)

查看更多
唯我独甜
3楼-- · 2019-04-16 00:20

Little confused what exactly you are looking for, in 0.8 we must specify the list of brokers in the metadata.broker.list property

    Properties props = new Properties();
    props.put("metadata.broker.list", "broker1:9092,broker2:9092");

from the kafka consumer example they say

The property, “metadata.broker.list” defines where the Producer can find a one or more Brokers to determine the Leader for each topic. This does not need to be the full set of Brokers in your cluster but should include at least two in case the first Broker is not available. No need to worry about figuring out which Broker is the leader for the topic (and partition), the Producer knows how to connect to the Broker and ask for the meta data then connect to the correct Broker.

By saying additions do you mean adding new node to your cluster ?

查看更多
登录 后发表回答