Distinct function

val ordersPerDay = ordersJoinOrderItems.
map(rec =>
rec._2._2.split(",")(1) + “,” + rec._1
record of(date,orderid).distinct
how come the distinct function is applied only with respect to orderid not order date


Can you please paste the entire code with sample output…

Distinct function returns unique combination of date and ordered. Means for each day there can be multiple of orders and each orders will have multiple of orderItems. If you list all records in order date and orderid, you will see one orderdate will have same orderid multiple times as each order will have multiple orderItems. So with distinct you will get one orderid for the orderdate.

1 Like