Introduction - Setup Development Environment – Download Spark compressed tar ball

As part of this topic, we will set up Spark in Windows Machine

  • Go to the link Spark Download
  • Choose the relevant version of Spark(this time it is 1.6.3)
  • Choose the Hadoop version(this time it is 2.6)
  • Click on the download link which will take to the downloads page
  • Copy the link and paste it in the Cygwin
  • wget
  • Copy the Spark folder from Cygwin to Local Directory
  • Download winutils from here , save the link into your local machine by creating a hadoop/bin folder
  • Set the path in the system variables
  • Change the permissions if still the spark-shell is not launched
  • Set the Spark path in system variables

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster

Not able to find Spark 1.6.3. So what to be done.