Which is efficient among fold and reduce

#1

Hi
I have a file call product.csv on hdfs. sample data:
101,Diary,109,9.95
102,Diary,110,1.25
100,Meat,2001,15.00
(there are 100s of records)

can anyone guide me wh one is efficient:

val products=sc.textFile("…/products.csv")
val prod_data=products.map{x => val z = x.split(",");(z(0).toInt,z(1),z(2).toInt,z(3).toDouble)}

case-1:
prod_data.reduce{case (x, y) => if (x._2 > y._2) x else y}

case-2:
prod_data.fold(0,“Dummy”,0,0)((acc1,acc2)=> if(acc1._3>acc2._3)acc1 else acc2)

It may be a basic qstn…but I am still learner…

#2

Hi

Found one issue with “fold”. If my dataset has only negative numbers then we should not use “fold”.

Here is the sample code:

val test = sc.parallelize(List((“test1”,-9),("test2,-4)))

test.fold(“Dummy”,0)((acc1,acc2)=> => if(acc1._2>acc2._2) acc1 else acc2)

Here is the output:
(String, Int) = (Double,0)

when I know that dataset has only negative numbers then I can change the code but when I donno “fold” seem to be not a good solution.

Can anyone please shed some light?