How to store the text file on the Master?

问题:

I am using Standalone clusters to run the ALS algorithm. The predictions are being stored to the textfile using:

saveAsTextFile(path)

But the text file is being stored on the clusters. I want to store the text file on the Master.

回答1:

That is expected behavior. path is resolved on the machine it is executed, the slaves. I'd recommend to either use a cluster FS (e.g. HDFS) or .collect() your data so you can save them locally on the master. Beware of OOM if your data is large.