How to integrate spark structured streaming with spark sql


#1

As I am new to spark, can any one give an example of spark structured streaming and spark sql integration example.

I want to take some CSV files on streaming basis from a directory using spark streaming and I need to join the input data with one of the table from hive using spark sql and want to write result in hive table.

I have searched for similar example but couldn’t find any. Only word count and Twitter streaming examples are available.So can any one please provide sample code for solving my problem

Note: I am using Spark version 2.3