Fix it ASAP - Continuous Issue while working on Spark Scala

apache-spark
scala

#1

below error is showing on screen while working on spark scala.

scala> 18/12/04 00:12:18 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 1
18/12/04 00:12:18 INFO ExecutorAllocationManager: Removing executor 1 because it has been idle for 300 seconds (new desired total will be 0)
18/12/04 00:12:19 INFO YarnClientSchedulerBackend: Disabling executor 1.
18/12/04 00:12:19 INFO DAGScheduler: Executor lost: 1 (epoch 0)
18/12/04 00:12:19 INFO BlockManagerMasterEndpoint: Trying to remove executor 1 from BlockManagerMaster.
18/12/04 00:12:19 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(1, wn03.itversity.com, 50261)
18/12/04 00:12:19 INFO BlockManagerMaster: Removed 1 successfully in removeExecutor
18/12/04 00:12:19 INFO YarnScheduler: Executor 1 on wn03.itversity.com killed by driver.
18/12/04 00:12:19 INFO ExecutorAllocationManager: Existing executor 1 has been removed (new total is 0)

It is coming very frequently and stopped working.


#2

@Akashivu Use sc.setLogLevel(“ERROR”) to stop displaying info logs in spark


#3

thank you Annapurna !


#4