How can I get the offset value in KStream

问题:

I'm developing a PoC with Kafka Streams. Now I need to get the offset value in the stream consumer and use it to generate an unique key (topic-offset)->hash for each message. The reason is: the producers are syslog and only few of them have ID's. I cannot generate an UUID in the consumer because in case of reprocess I need to regenerate the same key.

My problem is: the org.apache.kafka.streams.processor.ProcessorContext class expose an .offset() method that returns the value, but I'm using KStream instead of the Processor, and I couldn't find a method that returns the same thing.

Anybody knows how to extract the consumer value for each row from a Kstream? thanks in advance

回答1:

You can use mix-and-match DSL and Processor API via process(...), transform(...), and transformValues(...).

It allows you to access the current record offset similar to plain Processor API. In you case, it seems you want to use KStream#transform(...).