What is difference between hive and spark sql

what is difference between hive QL and spark sql

Hive QL in Spark (if that’s what you’re referring to) brings in Hive Context to Spark to perform sql querying. However, Spark has in-build memory to perform aggregations/querying as Spark is in-memory.

Also, if you are using Spark as ETL then you can query in Spark and ultimately write to Hive.

Let me know if this answers your question. @pb1879

no I am just referring to HIVE QL AND SPARK SQL… not calling hive in spark… but since you have pointed that out… how does hive context help in spark sql… will spark sql not be able to query hive metada without calling in hive context… please clarify

No spark sql will not be able to query hive metadata without calling hive context.