Orc support with sqlContext

Hi

I am able to use

sqlContext.read.orc("…")

without adding any databricks package. Can anyone please confirm if it is added in spark-shell?
It seems spark doesnt support “read.orc” as it supports “read.parquet”.

This is what I have read. It has native support for Parquet, json and jdbc. Not sure how did it read orc.

yeah…I am also surprised…I believe some package (databricks) added in spark-shell. Awaiting confirmation from @itversity

Package from databricks won’t be inbuilt in Hortonworks distribution (I think). Though you could add it extra (like for csv).

oops…are we using hortonworks in lab? I thought it is cloudera?
In hortonworks they have inbuilt package:

https://hortonworks.com/blog/bringing-orc-support-into-apache-spark/

itversity lab is hortonworks distribution.

oh…ok…
is there any way that we can identify ?

Refer to this answer.

I see ppl preparing for CCA175 which is on cloudera…so I was in an impression that it is cloudera :confounded:

@itversity can u plz clarify?

Yeah, but other than some minor difference here and there, how would it be different?
Mostly giving the same functionality. May be 5-10% (upper side) different than each other.

@spark - From CCA175 perspective you don’t see any difference when you use labs.itversity.com

thanks for clarifying

thanks for clarifying!!