Camel-SQL Why using StreamList seems to load all R

2020-07-23 06:53发布

问题:

Hi,

What am I trying to do ?

I am currently working on an ESB project (apache-camel + spring boot 2) where i read a MySQL table with more than 100 000 000 rows. I empty this table 1 row at a time, transform the row and send it to another database.

How am I doing this ?

Currently I use camel-sql to read the data

//edited .from(sql:SELECT * FROM mytable?outputType=StreamList&outpuClass=MyClass) .split(body()).streaming() .bean(mybean, "transform") .end()

Problem :

As I can't make a select * and get all 100M rows in my RAM because it's not possibly big enough, I thougth about using streams.

Therefore: It seems that using the StreamList as outputType still gets all rows at first and only then returns it as a "stream" (ResultSet).

Question

Can't we just use the property of PreparedStatement to really stream data from my database "one row at a time" rather than getting all of it at once and destroy my VM memory ?

Thanks.