Unknown configuration 'errors.deadletterqueue.

2019-08-03 19:18发布

问题:

I am trying to configure Sink Kafka Connect for Google Cloud PubSub Service.

Using following command to configure Kafka Connect:

curl 
-X POST 
-H 'Content-Type: application/json' 
-H 'Accept: application/json' -d '{ "name": "pubsub_test", 
"config": { "connector.class": "com.google.pubsub.kafka.sink.CloudPubSubSinkConnector", 
"tasks.max": "1", 
"topics": "kafka_test_topic",
"cps.topic": "cps_test_topic", 
"cps.project": "cps_test_project" } }' http://localhost:8083/connectors

In status, I have a following message:

{"name":"pubsub_test","connector":
{"state":"RUNNING","worker_id":"connect:8083"},
"tasks":[{"state":"FAILED","trace":"org.apache.kafka.common.config.ConfigException:
Unknown configuration 'errors.deadletterqueue.topic.name'\n\tat org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:91)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig.get(ConnectorConfig.java:117)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.get(ConnectorConfig.java:162)\n\tat org.apache.kafka.common.config.AbstractConfig.getString(AbstractConfig.java:126)\n\tat org.apache.kafka.connect.runtime.Worker.sinkTaskReporters(Worker.java:531)\n\tat org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:508)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:451)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:873)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:111)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:888)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:884)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n","id":0,"worker_id":"connect:8083"}],"type":"sink"}

回答1:

It looks like this is not specific to the Cloud Pub/Sub connector, but is a general issue with Kafka. Perhaps you are running a version of Kafka that does not have the fix.