As part of the testing of some sample program, I asked below:
I need expertise help here, especially at step III and step IV.
(Data for table (lastname, dayof the week, month of the week are need to generate using scala /spark api))
Please write a SparkSQL program to perform the following:
I. Create a database called sparktech.
sqlContext.sql(“CREATE DATABASE sparktech”);
II. Create a table called , with 3 columns called , ,
sqlContext.sql(“CREATE TABLE sparktech.fname (lastname string, day_of_week string, month_of_week string) STORED AS text”)
III. Insert 1000 rows of data into the table using Spark with constructed data from the same spark job. Specifically the data cannot be loaded in from file.
IV. Write a Spark to load the data from III, sort it, then write it to a csv file in HDFS.
Deeply, appretiate your help on this.