Agg function not working for multiple aggregations


#1

I am trying to show min, max, avg on same column using pyspark api by grouping on categoryID in products table in retail_db. but the result in showing the last agg function and ignoring the other two.

commands:
productsFilter = productsDF.filter(productsDF.productPrice < 100) productsFilter.groupBy(“productCatID”).agg({“productPrice”: “max”,“productPrice”: “min”,“productPrice”:“avg”}).show()

Screen shot:

image

could someone suggest a way to achieve this in single command instead of writing three separate commands for each max, min and avg.

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster