Verifying the solution

cca-175
scala

#1

problem stmt:
Instructions
Get the name of stocks displayed along with other information

Data Description
NYSE data with “,” as delimiter is available in HDFS

NYSE data information:

HDFS location: /public/nyse
There is no header in the data
NYSE Symbols data with tab character (\t) as delimiter is available in HDFS

NYSE Symbols data information:

HDFS location: /public/nyse_symbols
First line is header and it should be included
Output Requirements
Get all NYSE details along with stock name if exists, if not stockname should be empty
Column Order: stockticker, stockname, transactiondate, openprice, highprice, lowprice, closeprice, volume
Delimiter: ,
File Format: text
Place the data in the HDFS directory
/user/jkmasurkar/problem18/solution/
Replace whoami with your OS user name
End of Problem

my solution:
val nyse = sc.textFile("/public/nyse")

val symbols = sc.textFile("/public/nyse_symbols")

val nyseDF = nyse.map(n => (n.split(",")(0), n.split(",")(1), n.split(",")(2), n.split(",")(3), n.split(",")(4), n.split(",")(5), n.split(",")(6))).toDF(“stockticker”,“transactiondate”,“openprice”,“highprice”,“lowprice”,“closeprice”,“volume”)
nyseDF.registerTempTable(“nyse”)
val symbolsDF = symbols.map(s => (s.split("\t")(0), s.split("\t")(1))).toDF(“symbol”,“description”)
symbolsDF.registerTempTable(“symbols”)

val query = sqlContext.sql(“select distinct(n.stockticker),NVL(s.description,’’) as stockname, n.transactiondate, n.openprice, n.highprice, n.lowprice, n.closeprice, n.volume from nyse n left outer join symbols s on n.stockticker = s.symbol”)

Question: I am not sure if my solution is right because I am not able to find any records with empty value

please help.