How to remove blanks spaces in column on spark core

Dear Folks,

How to remove blanks in spark. Input File is tab separated and category column of file has some blanks how to remove those blanks in processing. Below output contains one blanks value (eg "34| "), i don’t want the blank value in output how to handle this

Code:

val youtubeRDD = sc.textFile(args(1) + “/youtubedata.txt”)
val category = youtubeRDD.map(line=>
{var YoutubeRecord = " “
val temp=line.split(”\t")
if(temp.length >= 3)
{YoutubeRecord=temp(3)}
YoutubeRecord})
val categoryMap = category.map(rec => (rec, 1))

val topCategoriesVideo = categoryMap.reduceByKey((total, value) => total + value).map(item => item.swap).sortByKey(false)

topCategoriesVideo.map(rec => rec._1 + “|” + rec._2).saveAsTextFile(args(2))

Output

908|Entertainment
862|Music
414|Comedy
398|People & Blogs
333|News & Politics
260|Film & Animation
251|Sports
137|Howto & Style
112|Travel & Events
95|Pets & Animals
80|Science & Technology
77|Autos & Vehicles
65|Education
42|Nonprofits & Activism
34|
32| UNA

Regards
venkat

Hi

change if condition like below

if(temp.length >= 3 && temp(3).size > 0) // here each category size should not be empty to avoid blank lines

1 Like

Thanks for the solution and it works fine as expected…

Regards
venkat