How to loop through the iterator

I created a RDD of below type using group by .RDD[(Int, Iterable[Float])] .I want to get min of that float.

scala> val pgroup= pdata.groupByKey
pgroup: org.apache.spark.rdd.RDD[(Int, Iterable[Float])] = ShuffledRDD[4] at groupByKey at :33

scala> pgroup.take(3).foreach(println)
(34,CompactBuffer(59.99, 59.99, 169.99, 169.99, 149.99, 149.99, 149.99, 129.99, 129.99, 129.99, 99.99, 99.99, 99.99, 149.99, 149.99, 149.99, 99.99, 99.99, 139.99, 139.99, 139.99, 139.99, 34.99, 99.99))
(52,CompactBuffer(170.0, 10.0, 170.0, 170.0, 170.0, 65.0, 130.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 26.0, 54.99))
(56,CompactBuffer(90.0, 75.0, 29.99, 90.0, 90.0, 39.99, 159.99, 90.0, 90.0, 159.99, 70.0, 29.99, 9.99, 50.0, 90.0, 30.0, 55.0, 90.0, 70.0, 90.0, 25.0, 70.0, 28.0, 28.0))

scala> val pRBK= pgroup.reduceByKey((k,v) => { if (k <v) k else v })
:1: error: ‘)’ expected but $XMLSTART$< found.
val pRBK= pgroup.reduceByKey((k,v) => { if (k <v) k else v })
^
:1: error: illegal start of simple expression
val pRBK= pgroup.reduceByKey((k,v) => { if (k <v) k else v })

use below code
val pRBK= pgroup.map(rec => (rec._1, rec.2.sum))
("
" is not visible after rec. so do necessary correction)

1 Like

Thank You . So i should not use reducebykey on top of an iterator .