Spark cache RDD don't show up on Spark History

Question

I am using Spark-1.4.1 in CDH-5.4.4.

I use rdd.cache() function but it show nothing in Storage tab on Spark History WebUI

Does anyone has the same issues? How to fix it?

Answer 1

Your RDD will only be cached once its been evaluated, the most common way to force evaluation (and therefor populate the cache) is to call count e.g:

rdd.cache() // Nothing in storage page yet & nothing cached
rdd.count() // RDD evaluated, cached & in storage page.