Does the spark submit mode depends on where the result is getting stored?
i.e., Running in Yarn mode for HDFS destination and Local to save in local FS
I tried to submit the python script which saves result in local FS. spark-submit works well in local mode. But when tried the same in yarn mode, it throws IOException saying Mkdirs failed to create some temp files in output directory
To save the results in local file system is it mandatory to submit spark job in local.