Lost executor driver on localhost: Executor heartb

问题:

I am debugging a spark application in local mode. Is it feasible to disable timeouts to avoid spark crashing in the middle of a debug session, without adverse effects?

Which timeout related parameters would you suggest changing for avoiding various timeouts without crashing spark altogether?

Currently I'm encountering this one, while debugging in IntelliJ:

Lost executor driver on localhost: Executor heartbeat timed out after 129006 ms

回答1:

Add these two into the mix:

 val spark = SparkSession.builder()
  .config("spark.network.timeout", "10000001")
  .config("spark.executor.heartbeatInterval", "10000000")