Regarding invalid syntax error

pyspark

#1

Hai,
I am getting the following error attached in the screenshot below when i try to convert a number into percentage in python.Can anyone tell me why this error occurs?


#2

As you are using map function along with the field, use split function.

thank you
Padmavathi


#3

@padmavathi As you said I used split function inside map function but i am getting the same error.


#4

Please send the total code. Here if the value in first column is of type int or float, we should typecast.

thank you
Padmavathi


#5

@padmavathi The complete code is as follows:

crime=sc.textFile("/public/crime/csv/crime_data.csv")
communitycodes=sc.textFile("/user/bigdatakarthik/crime/communitycodes.csv")
crimeMap=crime.map(lambda c:(c.split(",")[5],1))
crimecountbytype= crimeMap.reduceByKey(lambda x,y:x+y)
crimecountbypercentage=crimecountbytype.map(lambda c:(c.split(",")[0],int({:.2%}.format(c.split(",")[1]))))

Note: The error is shown on the colon i have given in the last line


#6

Hi,
Add quotes for the value which is being formatted.

crimecountbypercentage=crimecountbytype.map(lambda c:(c.split(",")[0],int("{:.2%}".format(c.split(",")[1]))))

Note: First Row of the file is header. As you are not using any method to map the fields excluding header Row, you won’t be able to get any preview for crimecountbytype also.

thank you
Pdmavathi


#7

@padmavathi When i try to execute the code you have given, i am getting the following error shown in the screenshot attached below:


#8

Use this command:
crimecountbypercentage=crimecountbytype.map(lambda c:(c[0],"{0:.2f}%".format(c[1])))