SQLContext is now working

apache-spark

#1

when i tried to convert my Data frame to temp table and wrote the sample query to get the record from the table , i am getting exception

res36.registerTempTable(“customer_table”)

scala> val finalDF = sqlContext.sql(“select * from customer_table limit 10”)
org.apache.spark.sql.AnalysisException: Table not found: customer_table;
at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.getTable(Analyzer.scala:305)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$9.applyOrElse(Analyzer.scala:314)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$9.applyOrElse(Analyzer.scala:309)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$resolveOperators$1.apply(LogicalPlan.scala:57)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$resolveOperators$1.apply(LogicalPlan.scala:57)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:69)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperato

operations that i have done:
val sqlContext =new org.apache.spark.sql.SQLContext(sc)
val hql =new org.apache.spark.sql.hive.HiveContext(sc)
val orders =sc.textFile("/public/retail_db/orders")
val customers =sc.textFile("/public/retail_db/customers")
val texasCustomer =customers.filter(x=>x.split(’,’)(7)==“TX”)
val customer_details =texasCustomer.map(x=>(x.split(’,’)(0),x.split(’,’)(1),x.split(’,’)(2))).toDF(“customer_id”,“customer_fname”,“customer_lname”)
val order_details =orders.map(x=>(x.split(’,’)(0),x.split(’,’)(2))).toDF(“order_id”,“customer_id”)
customer_details.registerTempTable(“customer_table”)
order_details.registerTempTable(“order_table”)
sqlContext.sql(“show tables”)

It is like the similar issue on the blog mentioned like https://community.hortonworks.com/questions/82480/orgapachesparksqlanalysisexception-table-not-found.html


#2

@loganathan You need to first import sqlContext.implicits._ before using toDF and register it as temp table and try your query.