Hi,
I am getting the error - “Not a data file”, when fetching the data from depts rdd using HiveContext.
Please guide how to fix this error.
Note - I am using spark 1.6.1
scala> import org.apache.spark.sql.hive.HiveContext
import org.apache.spark.sql.hive.HiveContext
scala> val sqlContext = new HiveContext(sc)
sqlContext: org.apache.spark.sql.hive.HiveContext = org.apache.spark.sql.hive.HiveContext@a32339a
scala> val depts = sqlContext.sql(“select * from departments”)
depts: org.apache.spark.sql.DataFrame = [department_id: int, department_name: string]
scala> depts.collect().foreach(println)
17/01/05 12:45:32 WARN spark.SparkContext: Requesting executors is only supported in coarse-grained mode
17/01/05 12:45:32 WARN spark.ExecutorAllocationManager: Unable to reach the cluster manager to request 2 total executors!
17/01/05 12:45:32 ERROR executor.Executor: Exception in task 0.0 in stage 1.0 (TID 6)
java.io.IOException: Not a data file.