@rahulabvp - Thanks for the clarification. I have one more doubt…could you please let me know how to write an RDD in CSV format?
I have 2 file tech.txt and sal.txt with below entries;
I am trying to join these 2 files based on first and last name and save the obtained result in csv format. But facing below error. Please help.
val tech = sc.textFile("/user/root/spark/technology.txt").map(x => x.split(","))
val sal = sc.textFile("/user/root/spark/salary.txt").map(x => x.split(","))
val joined = tech.map(x => ((x(0), x(1)), x(2))).join(sal.map(x => ((x(0), x(1)), x(2)))).collect()
joined.map(x => x.mkString("|")).saveAsTextFile("/user/root/empfile.csv")
error: value mkString is not a member of ((String, String), (String, String))
Thanks in advance