camel splitter parallel processing

Question

We have a service call which returns a list of ids with which we call another service which takes only one id at a time, so we are using the camel splitter with parallel processing turned to true. Now the call we make for the service is through a seda so we can put a timeout on it. This will cause the problem that parallel processing will not be parallel anymore since seda by default has only 1 concurrent consumers working on it.

Options:

1. put a ?concurrentConsumers=x on the seda
2. use direct rather than seda. (no timeout option?)
3. Any other option?

Of 1 and 2 which one would be preferable?

Answer 1

Firstly setting streaming to true on the splitter is useful for big messages. Which means it will split the input message in chunks. This reduces the memory overhead. So this will improve performance with large messages.

For a seda queue to truly process in parallel you need to set the seda route as follows.

<from uri="seda:report?multipleConsumers=true&amp;concurrentConsumers=16"/>

This will allow the route to use up to 16 threads for concurrent consumers. The seda component does not implement any kind of persistence or recovery, if the VM terminates while messages are yet to be processed they will be lost. If you need persistence, reliability or distributed seda, try using either jms or activemq

The direct component is synchronous and according to the documentation support for multiple consumers are deprecated. As of Camel 2.1: Direct endpoint does not support multiple consumers.

The other option is the vm component. The vm component differs from the seda component in that vm supports communication across CamelContext instances - so you can use this mechanism to communicate across web applications Essentially the vm component is an extension to the seda component.

So if if you need parallel processing the direct component is not useful. If you need to send message across CamelContent instances then use vm if not then use seda as described above.