Spark tar.gz does not have ec2 folder

I am trying to configure Spark cluster on EC2. But when I tried to download Apache Spark .tar.gz but it does not contain the ec2 folders. am I missing something ?.

What do you mean by ec2 folders?

@email2dgk It seems you might have downloaded Spark 2.x version. They removed managing ec2 scripts in ec2 folder of Spark binary from 2.x version. You can find the same here.

If you want to launch spark cluster using ec2 scripts, you can choose anyone of the following options:
Option-1:
If you are fine with Spark 1.6.x then download the latest stable version of Spark 1.6.x and you will see ec2 folder there

Option-2:
If you are insisted to go with Spark 2.x then you might have to do some workaround mentioned here. Iā€™m not sure whether this workaround works without any issues.

Hope this helps!

1 Like

@venkatreddy-amalla Thanks for the inputs. Let me try and let you know.

ec2 folder contains scripts and components which required to configure spark in EC2 cluster.