List all kafka topics

问题:

I'm using kafka 0.10 without zookeeper. I want to get kafka topics list. This command is not working since we're not using zookeeper: bin/kafka-topics.sh --list --zookeeper localhost:2181. How can I get the same output without zookeeper?

回答1:

Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one.

If you do not want to install and have a separate zookeeper server, you can use the convenience script packaged with kafka to get a quick-and-dirty single-node ZooKeeper instance.

Starting the single-node Zookeeper instance:

bin/zookeeper-server-start.sh config/zookeeper.properties

Starting the Kafka Server:

bin/kafka-server-start.sh config/server.properties

Listing the Topics available in Kafka:

bin/kafka-topics.sh --list --zookeeper localhost:2181

回答2:

to see that topic if we run the list topic command:

$ bin/kafka-topics.sh --list --zookeeper localhost:2181

To check if the data is landing in Kafka:

$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic twitterstream --from-beginning

回答3:

Kafka requires zookeeper and indeed the list of topics is stored there, hence the kafka-topics tool needs to connect to zookeeper too. kafka-clients apis in the newer versions no longer talk to zookeeper directly, perhaps that's why you're under the impression a setup without zookeeper is possible. It is not, as kafka relies on it internally. For reference see: http://kafka.apache.org/documentation.html#quickstart Step 2:

Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one

回答4:

The Kafka clients no longer require zookeeper but the Kafka servers do need it to operate.

You can get a list of topics with the new AdminClient API but the shell command that ship with Kafka have not yet been rewritten to use this new API.

The other way to use Kafka without Zookeeper is to use a SaaS Kafka-as-a-Service provider such as Confluent Cloud so you don’t see or operate the Kafka brokers (and the required backend Zookeeper ensemble).

For example on Confluent Cloud you would just use the following zookeeper free CLI command:

ccloud topic list

回答5:

For dockerized kafka/zookeeper

docker ps

find you zookeeper container id

docker exec -it <id> bash

cd bin

./zkCli.sh

ls /brokers/topics

回答6:

Kafka is a distributed system and needs Zookeeper. you have to start zookeeper too. Follow "Quick Start" here : https://kafka.apache.org/0100/documentation.html#quickstart

回答7:

Zookeeper is required for running Kafka. zookeeper is must. still if you want to see topic list without zookeeper then you need kafka monitoring tool such as Kafka Monitor Tool, kafka-manager etc.

回答8:

To read messages you should use:

kafka-console-consumer.sh --bootstrap-server kafka1:9092,kafka2:9092,kafka3:9092 --topic messages --from-beginning

--bootstrap-server is required attribute. You can use only single kafka1:9020 node.