How to setup/connect distributed Kafka brokers, pr

Question

I have setup Apache Kafka and confirmed that producers and consumers work on the localhost.

How to set up Kafka so that:

1. multiple producers feed messages into a broker on a network computer
2. many consumers on the network can consume the messages from the broker

I noticed the following line: zookeeper.connect=localhost:2181 in server.properties which is used to start the kafka server. If this is the setting, is the setting for what addresses it listens to or is it specifying the server's address/port is on the network?

Answer 1

The zookeeper is used internally by Kafka to coordinate the cluster (leader election). In versions of Kafka before 0.8, ZK was the exclusive store for consumer offsets (what is consumed so far), but from 0.8.1, I think, you can choose whether to store offsets in ZK or in a special Kafka topic called __consumer_offsets.

What you're interested in is advertised.host.name and advertised.port settings that Kafka exposes to clients (or "what addresses it listens to", as you put it).

Answer 2

It is the name of the zookeeper server that Kafka connects to. The documentation for the Broker configuration can be found here http://kafka.apache.org/documentation.html#brokerconfigs