Data Ingest - real time, near real time and streaming analytics - Spark Streaming – Get department wise traffic – Develop the program

Ramesh1 · November 13, 2019, 9:01am

gist.github.com

https://gist.github.com/dgadiraju/3c75ef0e3b077508b52e324c0d487ce8

StreamingDepartmentCount.py

from pyspark import SparkConf, SparkContext
from pyspark.streaming import StreamingContext

import sys

hostname = sys.argv[1]
port = int(sys.argv[2])

conf = SparkConf(). \
setAppName("Streaming Department Count"). \

This file has been truncated. show original

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

Click here for access to state of the art 13 node Hadoop and Spark Cluster