It seems there is a lot of confusion regarding spark related questions. Please find the clarification below:
- There will be a skeleton of code with the blanks. According to question we need to fill the blanks in script file (.py/.scala)
- Adjacent to it there will be a .sh file with somename.sh. Run this file with “./somename.sh” command.
- If you have filled with correct answer then the program will be executed and output will be saved in mentioned location, else you can see the errors in terminal.
- There is no difference between running script and running each and every transformation individually. Better to run script, because it is constructive, templates with few lines are already available. Saves your valuable TIME.
- It is not mandatory to use template. By using you can save so much time.
- You can launch shell and test spark transformations before filling blanks.
Let me know in case you need any more information.