How to configure IntelliJ to submit spark job on cluster

Hi,

How to configure IntelliJ to submit spark job on cluster

Can anyone please help here.

Regards,
Prem

1 Like

Why do you want to do that? We typically develop and test locally. Jars will be deployed on the cluster.

1 Like

We can not submit the spark jobs from the IDE’s .Only we can run on local mode through IDE’s.
We need to build a uber jar and copy it to cluster and do spark submit using yarn cluster mode .

In CCA 175 exam, would they provide IntelliJ, SublimeText or eclipse to develop the code.
While practicing we use IntelliJ but I confirmed with the people who has written CCA175 (old syllabus), they said that they provide nothing just a terminal.
How to handle this situation.
Please help!!