How to read data from mysql using pyspark


#1

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster


#2

Launch pyspark with below command

pyspark --jars /usr/share/java/mysql-connector-java.jar --master yarn --conf spark.ui.port=12643

and run the below code

mysql = sqlContext.read.format("jdbc").\
 option("url", "jdbc:mysql://nn01.itversity.com:3306/retail_db").\
 option("driver", "com.mysql.jdbc.Driver").\
 option("dbtable", "orders").\
 option("user","retail_dba").\
 option("password", "itversity").\
 load()

mysql.show()