I am trying to store the final result to HDFS getting below error message
#Get total revenue from order_items
1)
orderItemsRDD = sc.textFile("/user/gnanaprakasam/sqoop_import/order_items")
orderItemsMap = orderItemsRDD.map(lambda x: float(x.split(",")[4]))
orderItemsRevenue = orderItemsMap.reduce(lambda rev1, rev2 : rev1 + rev2)
orderItemsRevenue.saveAsTextFile("/user/gnanaprakasam/pyspark/totalRevenue")
‘float’ object has no attribute ‘saveAsTextFile’
Results => 34322619.930019915
- Same issue while trying to store using scala, also result differs. Any input on this ?
val orderItemsRDD = sc.textFile("/user/gnanaprakasam/sqoop_import/order_items")
val orderItemsMap = orderItemsRDD.map(x => (x.split(",")(4).toDouble))
val orderItemsRevenue = orderItemsMap.reduce((acc, value) => acc + value)
orderItemsRevenue.saveAsTextFile("/user/gnanaprakasam/sparkscala/totalRevenue")
value saveAsTextFile is not a member of Double
Result => 3.4322619930019915E7