Accumulators - Scala


Originally published at:

Let us explore accumulators It is important to perform some counts as part of application for unit testing data quality These counters cannot be global variables as part of the program Instead we need to use accumulator which will be managed by spark Accumulators will be passed to executors and scope is managed across all…