Can someone write eqivalent code for this in pyspark

pyspark
#1

val z=sc.parallelize(List(1,2,34,5,6),2)

z.aggregate(5)(math.max(,),+)

0 Likes

#2

Here’s the equivalent code for the above in pyspark.

z=sc.parallelize([1,2,34,5,6],2)
def seq_op(acc, val):
if(acc > val):
return acc
else:
return val

def comb_op(acc1, acc2):
return acc1+acc2

zero_val=5
z.aggregate(zero_val,seq_op,comb_op)

Hope it helps. Happy coding!

0 Likes

#3

Thank q very much for replying. However, by this time I have figured it and returned the code like this z.aggregate(5,max,lambda x,y:x+y).

0 Likes

closed #4
0 Likes