Why do we have separate services for Spark & Spark2 in Ambari?


I am taking ITversity lab for this example, I am using Spark 1.6.2 and I know it has linked to Spark service in Ambari

Then why do we have an additional service (Spark 2) in ambari? Does it connect to Spark 2.x by any chance? If so, why does it requires a separate service

Any clarification?

1 Like

I also have the same doubt. Can anyone clarify me?

My Understanding is as Spark 2.x and 1.6 are different versions, each has its own service in Ambari.

Spark 2.x is the latest version. But Spark 1.6.x is the most popular version.
Spark 1.6.2 is generally available while Spark 2.x is in beta for future exploration.

You should focus on 1.6.2 for now in pursuit of preparing for the certifications. If you want to learn Spark 2.x, still you can do so.

PS: I will be creating Spark 2.x content in couple of months.

Someone raised the similar question in Cloudera Community and a Cloudera Employee responded to that… it answers my question

“Spark 2.x is a separate parallel install so as to not replace Spark 1.x”