Is there an easy way to install Kafka on Windows?

问题:

I'm trying to install Kafka message queue on Windows for testing purpose (not for production :D).

I only found this article on how to install Apache Kafka 0.8 on Windows: http://janschulte.wordpress.com/2013/10/13/apache-kafka-0-8-on-windows/

That's cool but:

1. It's outdated
2. It's way too complicated...

Someone know an easy way to achieve that?

Thanks.

回答1:

Ok, it's finally not complicated :)

The only steps are:

1. Download Kafka and uncompress it somewhere nice (let's say C:/Kafka)
2. Install Cygwin

3. Edit \bin\kafka-run-class.sh and at the end of the file, change

   exec $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@"

   to

   exec java $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp `cygpath -wp $CLASSPATH` $KAFKA_OPTS "$@"

4. In Environment Variables, Add java to your Path System Variable:

   

That's it.. you can now run ZooKeeper and Kafka servers and start playing with topics and stuff..

回答2:

The Steps are as follows-

1. The Java Environment variables should be set.
2. Got to the Apache Kafka downloads page and download the Scala 2.12 kafka_2.12-0.10.2.1.tgz
3. Unzip it.

4. Open cmd prompt and start zookeeper-

   C:\kafka_2.12-0.10.2.1>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

5. Open a new command prompt and start the Apache Kafka-

   C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-server-start.bat .\config\server.properties

6. Open a new command prompt and create a topic with name javainuse-topic, that has only one partition & one replica.

   C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic javainuse-topic

7. Next Open a new command prompt and create a producer to send message to the above created javainuse-topic and send a message - Hello World Javainuse to it-

   C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic javainuse-topic

   Hello World Javainuse

8. Finally Open a new command prompt and start the consumer which listens to the topic javainuse-topic we just created above. We will get the message we had sent using the producer

   C:\kafka_2.12-0.10.2.1>.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic javainuse-topic --from-beginning

Detailed steps including screenshots and video are available in this blog- Getting started with Apache Kafka

回答3:

A more current answer for the benefit of anyone else still wondering about this, I downloaded the binary and everything worked right out of the box. The source version, however, didn't work.

回答4:

These are the steps I followed to run kafka on Windows

1. Install Zookeeper first (I downloaded v3.3.6) zookeeper-3.3.6.tar.gz
2. Extract Zookeeper and run this command in powershell/cmd \zookeeper-3.3.6\bin> .\zkServer.cmd Now this should up a Zookeeper instance on localhost:2181
3. Download Kafka binary version (I downloaded v0.10.0.1)kafka_2.10-0.10.0.1.tgz
4. Extract Kafka, time to modify some configs
5. Inside Kafka extraction you can find .\config\server.properties
6. In .\config\server.properties replace log.dirs=c:/kafka/kafka-logs
7. Note: Make sure to create those folders in relevant paths
8. Happy news: Now Kafka ships with windows .bat scripts, You can find these files inside ./bin/windows folder
9. Start powershell/cmd and run this command to start Kafka broker .\bin\windows\kafka-server-start.bat .\config\server.properties
10. DONE!, Now you have a running Zookeeper instance and a Kafka broker.

回答5:

You will probably find that Kafka comes with Windows .bat files under the bin folder to help you run Kafka under windows. However, those bat files are not updated since version 0.8.0 and some of Kafka's logic has changed (regarding topics, etc.).

I have tried two ways to solve this problem. One is to use Cygwin or MinGW to simulate Linux environment and run the Linux shell script, but there would be tons of other problems related to path names. The other solution, which is simpler and come with less trouble, is to use the corrected version of Windows bat files.

Please refer to this blog post.

回答6:

In version 0.9.0. there are scripts in "bin" folder for windows. You can use them.

回答7:

This is an old thread but for someone reading this in 2018 I suggest running kafka in docker. One of many tutorials about this: https://iteritory.com/kafka-docker-image-installation-usage-tutorial-windows/

回答8:

Yes, I had configured ZOOKEEPER and Apache Kafka on my Windows machine. For ZOOKEEPER, simply configure ZOOKEEPER_HOME and set the path also and then rename the zoo.sample.cfg to zoo.cfg and also change the path for dataDir in the zoo.cfg

For Apache Kafka, simply unzip and start the server from bin folder and start the Producer/Consumer from windows folder under the Kafka directory

回答9:

1. Download any binary version(I've downloaded kafka_2.11-0.10.2.0) of Kafka from https://kafka.apache.org/downloads
2. Extract it to any folder(I've extracted to "C:\Kafka")
3. Open Command Prompt
4. Go to the folder that you extract Kafka(C:\Kafka\kafka_2.11-0.10.2.0)
5. Run this command .\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties to start Zookeeper
6. Run this command .\bin\windows\kafka-server-start.bat .\config\server.properties to start Kafka

Now it works!

回答10:

With Chocolatey and Powershell:

1. chocolatey.exe install kafka -> installs Kafka into C:\ProgramData\chocolatey\lib\kafka\tools\kafka_2.11-1.0.0\ - your version might differ of course
2. add C:\ProgramData\chocolatey\lib\kafka\tools\kafka_2.11-1.0.0\ to your powershell path using notepad $PROFILE

After restarting powershell kafka-console-consumer should work as a command.

You also need to install ssl.properties with keystore and truststore. Put them e.g. into C:\ProgramData\kafka and put that into ssl.properties (please note the escaping of the backslashes):

security.protocol=SSL

ssl.truststore.location=C:\\ProgramData\\kafka\\kafka-truststore.jks
ssl.truststore.password=PASSWORD

ssl.keystore.location=C:\\ProgramData\\kafka\\kafka-keystore.jks
ssl.keystore.password=PASSWORD
ssl.key.password=PASSWORD

client.id=console-test

Now things like kafka-consumer-groups --bootstrap-server SERVERNAME:9094 --command-config C:\ProgramData\kafka\ssl.properties --list should first pop up a firewall warning which you'd need to accept and then output a list of groups.