Sc & sqlContext commands not working in scala


#1

Hi,

I am having issues to read contents of file in scala using sc, When i type the command sc in scala console i am getting an error " not found value of sc"


#2

Spark APIs are not available in Scala by default.

Here are the steps you need to perform (if you are using sbt):

  • Update build.sbt with relevant spark dependency
name := "spark2demo"

version := "0.1"

scalaVersion := "2.11.12"

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.0"
import org.apache.spark.{SparkConf, SparkContext}
val conf = new SparkConf().setMaster("local").setAppName("Testing")
val sc = new SparkContext(conf)

#3

@Hdpins If you plan to just learn Spark then please download spark and use spark-shell. I believe you use REPL for scala.