Reuse a cached Spark RDD

2019-02-25 16:06发布

Is there a possibility in Spark to re-use a cached RDD in another application (or in another run of the same application)?

JavaRDD<ExampleClass> toCache = ... // transformations on the RDD
toCache.cache();                    // can this be reused somehow in another application or further runs?

1条回答
我只想做你的唯一
2楼-- · 2019-02-25 16:29

No, Spark RDD cannot be used in other application or in another run.

You can connect Spark with for example Hazelcast or Apache Ignite to save RDDs in memory. Other application will have possibility to read data saved in first application

查看更多
登录 后发表回答