Pyspark hivecontext

from pyspark import HiveContext
from pyspark import SQLContext
sqlContext = HiveContext(sc)
depts = sqlContext.sql(“select * from departments”)

Also I have my own database

Could you please help !!

to use this big data labs is highly frustrating now :frowning:

1 Like

@Sayali_Mahajan -

/user/sayli09 is HDFS user space allocated for you. It’s not a database.

I followed below steps and could able to see the data from orders table.

hive
use sayali09;
show tables;

I see only orders table under sayali09 database.

from pyspark import HiveContext
from pyspark import SQLContext
sqlContext = HiveContext(sc)
depts = sqlContext.sql(“select * from sayali09.orders”)
for i in depts.take(10): print(i)

1 Like

Hello Sayali,

We will not be able to respond in real time. @gnanaprakasam have responded to your query.
If you still have issues, let us know.

I saw your email about spark 1.2. We do not need to set up as the syllabus is changed. Also list the issues which are making you frustrated while using the lab, we will work on them as long as they are technical issues with respect to lab.

You need to follow this playlist for updated syllabus, but we do not have content yet on Python -