My jobs often hang with this kind of message:
14/09/01 00:32:18 INFO spark.MapOutputTrackerMasterActor: Asked to send map output locations for shuffle 0 to spark@*:37619
Would be great if someone could explain what Spark is doing when it spits out this message. What does this message mean? What could the user be doing wrong to cause this? What configurables should be tuned?
It's really hard to debug because it doesn't OOM, it doesn't give an ST, it just sits and sits and sits.
This has been an issue from Spark at least as far back as 1.0.0 and is still ongoing with Spark 1.5.0
