I have a web service hosted which allows to pull records in batch. This web service takes starting record number (ROWID) and page size(800 max) as parameters. There could be 50-60k records to pull from this service and call another web service to post all these data again in chunk without persisting data in between.
How could I use Spring Batch to pull the records page by page (chunking) by calling web service and how do I post same records to another web service.
I was able to do this using Spring-Integration batch but for large set of data, i am not sure whether Spring-Integration is ideal way of doing when we have Spring-Batch for processing large set of data.
Spring Batch doesn't have a web service ItemReader per say. That being said, if you create a custom ItemReader that extends AbstractPagingItemReader the paging logic itself should be taken care of for you (you implement the doReadPage() method that handles getting a page of data, the super class handles keeping track of what page you're on, etc).
For the ItemWriter side of things, if you have a client you want called, you can use the ItemWriterAdapter. This will call a method on a java object, passing it each item within the list passed to the ItemWriter#write(List items) method. Otherwise, you'll need to write your own.
In either case, the custom code you'll need should be minimal.
