apache storm reliablity timeout configuration

Question

I have a nodejs->kafka>storm->Mongo deployed in Linux Ubuntu. Everything is normal originally. Then I changed the method in storm worker which makes storm worker process message very slow, around 1 minute per message, I notice the message is sent again and again from storm. I revert back to original method, everything is fine. (original method process time is 90ms per message).

I guess this is Storm reliability come into player. When message is not acknowledged, or time out, it sends message again.

If my guess is right, how to configure this timeout?

If my guess is wrong, why same message is sent twice or three times?

Answer 1

You can set the timeout via configuration parameter Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS. See https://storm.apache.org/javadoc/apidocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS

The default value is 30 seconds, see defaults.yaml here: https://github.com/apache/storm/blob/master/conf/defaults.yaml

# maximum amount of time a message has to complete before it's considered failed
topology.message.timeout.secs: 30

When a tuple fails, it should show up in Storm UI and should be logged, too (maybe you need to adjust log level). So you can double check if a tuple times out or not.