Using sbt for spark streaming questions for cca 175

cca-175
spark-streaming

#1

Hello All,

I have a small doubt regarding spark streaming questions for the cca 175. In the labs we have been packaging related jar files with sbt and scp’ing to the hdp cluster. Would we be able to do this with cdh5 cluster at the time of exam?

If not, i see there will already be intelliJ installed in the cluster for us at the time of the exam, but it does not look like they provide us with sbt. can we import the sbt plugin for packaging the jar files in the cluster itself.

Can somebody please verify this to me ?

Regards
venkat


#2

In the exam, you don’t need to make a jar files with sbt. For the given problem, you need to write 4 to 10 lines of code and get the result. The sublime text editor will be provided to write and format the code, not IntelliJ.


#3

But we were noticing in the udemy lectures that we cannot use the spark shell for running spark-streaming applications. Why would it work in the exam when it wasnt working in our labs ?


#4

Plus, if we are using, flume integration, we have to use the spark submit to provide those additional libraries in which case we have to provide our code as a jar too, how do you deal with that scenario then without sbt ?