Deploying a Spark application in AWS

Hi,

Currently i am undergoing one interview evaluation process for Data Engineer. As part of this i got a use case to make spark application for calculating surge pricing mechanism for rides between the same source and destination in system like UBER or OLA.

They have given me an AWS console access and asked to setup everything and deploy the application to it. I am new to AWS. Dont know how to start with it.

Could you pleas guide me on this

Regards,
Mohamed

1 Like

My initial thoughts -
Use Kafka to create rides as events.
Use Spark streaming to process events from Kafka and calculate surge pricing.
This link might help to get started - https://aws.amazon.com/blogs/big-data/real-time-stream-processing-using-apache-spark-streaming-and-apache-kafka-on-aws/