I am attending interviews, but i stuck with the fallowing questions. Please reply if you anybody know answers.
- How many partitions does Spark default create for a large data files ?
- If spark creates default number of partitions of 5, why we need to use coalesce and repartion to increase the number of partition.
- Suppose i have 4 partitions with 1 GB of parition, and in spark-submit i given 5 executor core with 4GB of exeutor memory , how spark will perform the task?
Please reply if know the answer
Thanks and Regads