Apache spark Real time projects


#1

Hi, Can anyone share some realtime projects explanation/scenarios which are useful for the interviews. it is great help for the people who is looking job on spark.

At least it is good high level flow(E2E Flow) with spark sql and spark streaming.


#2

Yes. I am also looking for this. Any one please help


#3

I am about to start a project that uses Kafka to Spark streaming on realtime data. I myself am not expert at spark and I believe I will need to write everything using scala. Maybe I can show what I am doing to see if the approach is correct.


#4

Kindly share the details of the project that you were working on


#5

Kindly Share the Details to the below EmailId :- esakkisankart@gmail.com


#6

Hello,
I would be more than happy to assist you on this project if it’s okay. It would great if you share the repository on git or mail it to me on akhilanand.bv@gmail.com


#7

I cannot post any code to git since company would not be happy but what I can say is this project gets real time data from a JMS service.

I can say for sure most jobs you get currently most likely be running JMS (Java Messaging Service) and want to convert to big data like Kakfa/Spark stream.

First hurdle is using Scala/Spark to parse out a given XSD schema of data coming in off JMS/Kafka topics. I have an XSD for my data and use Maven plugin to create my POJOs. I am not sure how to do this under scala yet.

Perhaps we, on Itversity can do something similar. We can use any XSD file, like stock market data or something and work with that. From there we can generate sample data or use what is in Durgas cluster to work with.


#8

Kindly Share the Details to the below EmailId :- miteshgala92@yahoo.com


#9

Hello,
We are using Spark in Multiple ways were there are lot of scenarios

  1. DataSource -> Spark Streaming -> HDFS
  2. kafka -Spark Streaming -> HDFS
  3. Kafka -> Spark Streaming -> Hive
  4. Kafka -> Spark Streaming -> Hbase
    All are in Java

#10

hi… could you please share the details of the project that you have mentioned above…here is my email id: tkhan.kiit@gmail.com


#11

Hello,

Can you please share the details to this mail id gopisainadh25@gmail.com?

Thank you.


#12

I am looking for a spark real time project explanation from requirement analysis to building test cases and deploy the code into PROD.
Please help me if anyone of you know any institute in Hyderabad.