Advice on Integration Flow Design

Question

I am planning to implement an integration flow as below:

IntegrationFlows.from(httpInboundGateway)
    .transform(transformer-rest-api-1)
    .transform(transformer-rest-api-2)
    .handle(jdbc-outbound)
    .handle(http-outbound-gateway-1)
    .get(); 

The requirements that I want to fulfill are:

1. to make this run in parallel threads as much as possible
2. persist message at every end-point
3. make very endpoint as rest-api (to make the flow scalable)

Does it make any sense to make the flow reactive? If so how to go about it? How do I log at every step? (Does wiretap help?), and Finally, can you please provide a concrete example that of a simple implementation for above in java dsl?

Answer 1

1. To make it parallel you need to consider to use an ExecutorChannel in between endpoints:

   IntegrationFlows.from(httpInboundGateway)
      .channel(c -> c.executor(myTaskExecutor()))
      .transform(transformer-rest-api-1)
   

and so on.

2. However since you would like to have a persistence option, you need to consider to use a QueueChannel with a persistent MessageStore: https://docs.spring.io/spring-integration/docs/current/reference/html/system-management-chapter.html#message-store. Each endpoint then has to be supplied with the poller options, including a tasExecutor for parallel processing:

   .channel(c -> c.queue(jdbcMessageStore(), "queue1Channel"))
          .transform(transformer-rest-api-1, 
                        e -> e.poller(p -> p.fixedDelay(100).taskExecutor(myTaskExecutor())))
   

3. To make endpoints as REST API you just need to use Http.outboundGateway(). Or for reactive variant - WebFlux.outboundGateway() in the .handle() instead of transform(). Or just go ahead in the existing .handle(http-outbound-gateway-1) in your flow

4. To make flow reactive you need to use a .channel(c -> c.flux())), but you'l lose a persistence on the matter.

5. To log each step, there is a .log() operator to be placed in between endpoints.

6. Your requirements are not clear to share some sample...