Getting SPARK-5063 err

bigdatalabs

#1

File “/usr/hdp/current/spark-client/python/pyspark/rdd.py”, line 206, in getnewargs
"It appears that you are attempting to broadcast an RDD or reference an RDD from an "
Exception: It appears that you are attempting to broadcast an RDD or reference an RDD from an action or transformation. RDD transformations and actions can only be invoked by the driver, not inside of other transformations; for example, rdd1.map(lambda x: rdd2.values.count() * x) is invalid because the values transformation and count action cannot be performed inside of the rdd1.map transformation. For more information, see SPARK-5063.


#2

Can you send the complete code?