*****Spark-shell is throwing errors on executing actions


#1

Support folks,

Please can you check if any issues with the cluster. I am seeing many errors related to “Caused by: java.nio.channels.ClosedChannelException”. I am running below commands.
spark-shell --master yarn
–conf spark.ui.port=12345
–num-executors 10
–executor-memory 3GB
–executors-core 2
–packages com.databricks:spark-avro_2.10:2.0.1

sc.textFile("/public/randomtextwriter").map(line=>line.replaceAll("[^a-zA-Z0-9//s+]","")).flatMap(line=>line.split(" “)).map(line=>(line,1)).reduceByKey(+,8).map(line=>line._1+” "+line._2).toDF(“words”, “count”).show()

hdfs:///spark-history/application_1525279861629_26539


#2

Hi @rr4074,
What you are trying to achieve ?