Hi Durga Sir,
While trying to run query in SQLContext from pyspark, i’m facing some issue when i’m trying to invoke sqlContext.load() method along with its parameters. Below is the screenshot of it…
Please guide me how to rectify this as it’s saying load method is in some other file. Your help is needful here.
Regards,
Saquib Razzaque
Can you try removing double quotes i.e. " from jdbcurl in load function. (url = jdbcurl)
launch the pyspark as below:
pyspark --driver-class-path /usr/share/java/mysql-connector-java.jar
( first check if there is a connector in the following location mostly it should be available )
then execute these commands:
from pyspark.sql import SQLContext
sqlContext = SQLContext(sc)
jdbcurl = "jdbc:mysql://nn01.itversity.com:3306/retail_db?user=retail_dba&password=itversity"
df = sqlContext.load(url=jdbcurl, source=“jdbc”,dbtable=“departments”)
for i in df.collect():
print i
Above, please note that url should be the first parameter. Please let me know if still run into any issues. Thanks!
Which version of spark you are using?
Hi ,
I am also getting the same issue . Have you resolved the issue ?
I am using spark 1.3.0 version.
Please Help…
Thank you very much … It worked …
