How to read logs from file in kafka?

2019-09-21 03:11发布

问题:

I want to read Apache logs in kafka and then further process in to Spark Streaming.I am new to kafka. As far as I have understand I have to write a producer class to read logs file.

回答1:

You can do so by creating a connector which sources each line of the log file into the Kafka topic. Check out the example here:

https://docs.confluent.io/current/connect/devguide.html#connect-developing-simple-connector