Apache Spark 2.x – Data processing - Getting Started - Using ITVersity labs

Using ITVersity labs

At ITVersity we rovide multi-node Spark Cluster. Let us understand some important details with respect to ITVersity labs.

  • Click here to visit our labs.
  • We have 10+ node cluster in which Spark is integreated with Hadoop ecosystem
  • It has a total capacity of 80+ cores and 400+ GB Memory and severak terabytes of storage.
  • The cluster is built using Hortonwork’s and cluster can be reviewed using Ambari.
  • It has both Spark 1.6.3 and Spark 2.3.0
  • You can launch spark 2 either by exporting SPARK_MAJOR_VERSION to 2 or by using spark2 related commands.
    • spark2-shell(Spark 2 with Scala)
    • pyspark2(Spark 2 with python)
    • spark2-submit(Spark 2 submit command to submit Spark 2 jobs)

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster