Grouping/summarizing spring-batch records

2020-03-31 03:52发布

I'm new to spring-batch and I want to use it to implement the following proces:

read: a list of items from webservice
process: groups these items
write: these groups back to webservice

Example:

Input:
transactionID, store, amount
1, FooStore, 2.50
2, BarStore, 19.99
3, FooStore, 12,49

Output:
totalsID, store, amount
1, FooStore, 14.99
2, BarStore, 19.99

Is this even possible using spring-batch, am I missing an important step, or does this conflict with the concept of batch processing to such an extent that I should look for a different solution?

I have found a possible solution, that is by using the writeFooter method, but that seems silly because I fear I'll lose a lot of the benefits of using spring-batch like the error recovery and memory management. Plus as far as I can see the footer is only supported if the output is written to a file, not send to a webservice.

1条回答
够拽才男人
2楼-- · 2020-03-31 04:28

Write a job with two steps:

  1. Read every record and from input then, in your own ItemWriter.write(), update a summary table
  2. Dump your summary table to text file (or webservice)

You can also try to reach the goal writing a single SQL statement using aggregate function like SUM() and COUNT() then, with a single-step job, read every record generated from custom SQL and write directly to you text file (or webservice).

查看更多
登录 后发表回答