what is difference between hive QL and spark sql
Hive QL in Spark (if that’s what you’re referring to) brings in Hive Context to Spark to perform sql querying. However, Spark has in-build memory to perform aggregations/querying as Spark is in-memory.
Also, if you are using Spark as ETL then you can query in Spark and ultimately write to Hive.
Let me know if this answers your question. @pb1879
no I am just referring to HIVE QL AND SPARK SQL… not calling hive in spark… but since you have pointed that out… how does hive context help in spark sql… will spark sql not be able to query hive metada without calling in hive context… please clarify
No spark sql will not be able to query hive metadata without calling hive context.