Spark job is getting stuck at one stage displaying a warning


#1

Spark job is getting stuck at a particular stage by displaying the below warning statements.

19/02/01 02:19:35 WARN SparkContext: Requesting executors is only supported in coarse-grained mode
19/02/01 02:19:35 WARN ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!
19/02/01 02:19:36 WARN SparkContext: Requesting executors is only supported in coarse-grained mode
19/02/01 02:19:36 WARN ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!
19/02/01 02:19:37 WARN SparkContext: Requesting executors is only supported in coarse-grained mode
19/02/01 02:19:37 WARN ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!
19/02/01 02:19:38 WARN SparkContext: Requesting executors is only supported in coarse-grained mode
19/02/01 02:19:38 WARN ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!
19/02/01 02:19:39 WARN SparkContext: Requesting executors is only supported in coarse-grained mode
19/02/01 02:19:39 WARN ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!

Please help me if someone has an idea.

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster