Hi,
What am I trying to do ?
I am currently working on an ESB project (apache-camel + spring boot 2) where i read a MySQL table with more than 100 000 000 rows. I empty this table 1 row at a time, transform the row and send it to another database.
How am I doing this ?
Currently I use camel-sql to read the data
//edited
.from(sql:SELECT * FROM mytable?outputType=StreamList&outpuClass=MyClass)
.split(body()).streaming()
.bean(mybean, "transform")
.end()
Problem :
As I can't make a select * and get all 100M rows in my RAM because it's not possibly big enough, I thougth about using streams.
Therefore: It seems that using the StreamList as outputType still gets all rows at first and only then returns it as a "stream" (ResultSet).
Question
Can't we just use the property of PreparedStatement to really stream data from my database "one row at a time" rather than getting all of it at once and destroy my VM memory ?
Thanks.