It’s easy to create a data-frame using Spark-SQL and process, but daily new data loading into database/table. So that I want to use Spark Streaming, means when the data is loaded in MySQL, intently load it in Spark Streaming and process in Spark. To do it is it possible to get MySQL/RDBMS data using Spark Streaming?
@sunny407 Can you please let us know the use case that you want to achieve ?
I think there is no straight way to do this . We need to implement custom receiver to steam data from RDBMS . There is a concept called custom receiver in spark .
You can try that . This is just my thought process .
Please let me know if any one having a better approach .
This a old question posted already on databricks forum . See the response .