Publish null/tombstone message with raw headers

2019-09-06 13:49发布

问题:

I am building a Spring Cloud Stream Kafka processor app that will consume raw data with a String key and sometimes a null payload from a Kafka topic. I want to produce to another topic a String key and the null payload (known as a tombstone within Kafka). In order to use raw headers on the message, I need to output a byte[], but if I encode KafkaNull.INSTANCE into a byte[] it will literally output a String of the object hashcode.

If I try to send anything other than a byte[], I can't use raw headers.

What is the correct way to do this? Where would the headers even be if the payload is null? I set producer.headerMode=embeddedHeaders on deploy and that didn't seem to make a difference, still have the hashcode as the payload.

回答1:

You can use the latter introduced useNativeEncoding property (which overlaps the raw headerMode in that it skips headers) but allows you use a native Kafka serializer.

See http://docs.spring.io/autorepo/docs/spring-cloud-stream-docs/Brooklyn.BUILD-SNAPSHOT/reference/htmlsingle/#_producer_properties or http://docs.spring.io/autorepo/docs/spring-cloud-stream-docs/Chelsea.BUILD-SNAPSHOT/reference/htmlsingle/#_producer_properties