Set SQLContext = HiveContext CCA175

Hi All,

On lab (provided by itversity) the spark version is 1.6.2
there sqlContext = hivecontext

scala> sqlContext
res36: org.apache.spark.sql.SQLContext = org.apache.spark.sql.hive.HiveContext@3dcc59f5

Due to this

if we try to read Avro file store in HDFS location
val file1 = sqlContext.read.avro(“some-avro-file”)
and then store this file as table in hive

file1.saveAsTable(“hiveDB.TableName”)

is valid. If you check this table in that particular hive DB it present.

However, in exam they will provide spark 1.6
so, my doubt is how to set

sqlContext pointing to hive

so that

org.apache.spark.sql.SQLContext = org.apache.spark.sql.hive.HiveContext@3dcc59f5

Please help!!

I have not given the exam, but by default when you launch Spark, sqlContext will point to Hive which can be checked as you did here. If not you can do
import org.apache.spark.sql.hive.HiveContext
val sqlContext =new HiveContext(sc)
Here instead of sqlContext you can use any name e.g. hiveContext, but in query you need to use same name as you have given.

2 Likes

Thank You N_Chakote!!

Also what is the error you are getting in quickstart vm?