Error while using takeOrdered - Spark Scala

scala

#1

HI,
Getting error when I tried to run following script… getting similar error in def when tried to run in REPL:

import math.Ordering.Float._
import io.Source
def topNProducts(line : Iterator[String],N:Int,catId:Int=0):Iterator[String]={
var prod = line
if(catId!=0)
prod=line.filter(row=>row.split("\001")(1).toInt==catId)
val prices = prod.map(pr=>pr.split("\001")(4).toFloat)
val topN = prices.takeOrdered(N)(Ordering[Float].reverse.on(p=>p))
val result = prod.filter(row=>row.split("\001")(4).toFloat<=topN)
result
}
val products = Source.fromFile("/user/hive/warehouse/jb_mk.db/products/").getLines
val results = topNProducts(products,5,41)
results.collect.foreach(println)

The error is
error: value takeOrdered is not a member of Iterator[Float]
val topN = prices.takeOrdered(N)(Ordering[Float].reverse.on(p=>p))

trying to implement Top N priced products example from the class…

What’s wrong with the code?

thanks,
regards,
JN


Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster