Unable to read the data from kafka using spark

#1

Hello,

I am trying to read the message from kafka topic suing the spark streaming, but I am geting below error message.
19/05/26 03:43:11 ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting receiver 0 - java.lang.ClassCastException: java.util.HashMap cannot be cast to java.lang.String
at org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:96)

Below is my code:
from pyspark import SparkConf, SparkContext
from pyspark.streaming import StreamingContext
from pyspark.streaming.kafka import KafkaUtils
import pprint

conf= SparkConf().setAppName(“streaming test”).setMaster(“yarn-client”)
sc=SparkContext(conf=conf)
ssc=StreamingContext(sc,10)

topic = “newone”
broker = {“metadata.broker.list”: “nn01.itversity.com:2181, nn02.itversity.com:2181, rm01.itversity.com:2181”}
direct = KafkaUtils.createStream(ssc,broker,“test”,{topic:1})

direct.pprint()
ssc.start()
ssc.awaitTermination()

could you please help me with this issue. Thank you.

0 Likes