Overview of YARN


Originally published at: http://www.itversity.com/topic/overview-of-yarn/

Let us see brief overview of YARN and the role it play in Spark Make sure you have virtual machine of Cloudera or Hortonworks are setup or have access to the lab Let us revisit Spark architecture Run a sample job Validate files in HDFS hadoop fs -ls /public/randomtextwriter/part-m-00000 Launch spark in scala context using spark-shell YARN…