Accumulators

Originally published at: http://www.itversity.com/topic/accumulators/

Introduction In this topic we will see details about accumulators It is important to perform some counts as part of application for unit testing data quality These counters cannot be global variables as part of the program Instead we need to use accumulator which will be managed by spark Accumulators will be passed to executors…