Connecting spark with AWS Redshift

Connecting spark with AWS Redshift
3.0 1


Hi All,

Could you please let me know how to connect Redshift with Spark and Scala in eclipse IDE .

Could you please let me know all the import statements and permissions to connect to redshift from spark .

Please provide me an example if possible.


This link might help and give you idea:


Hi Vinod,

Thanks for providing the document.

I have tried in the below way to connect to the redshift cluster but getting session timeout error.
.option(“url”, “jdbc:redshift://”)
.option(“dbtable”, “vfdastg.ipstoremaster”)
.option(“tempdir”, “s3n://vf-dtc-da-dev-emr-kiran-reddy/Testing”)
.save() *

any thing we should add.

Thanks in advance.


No I don’t have idea on this, But can you paste the output also to get some more relevant details.


Hi Vinod,

Thank you.

I have one more issue.

How to write a dataframe as a KMS Encrypted file in s3 bucket using spark, I have given the below parameters.

sc.hadoopConfiguration.set(“fs.s3n.awsAccessKeyId”, “---------”) sc.hadoopConfiguration.set(“fs.s3n.awsSecretAccessKey”, “-------”) sc.hadoopConfiguration.set(“fs.s3a.server-side-encryption-algorithm”, “SSE-KMS”) sc.hadoopConfiguration.set(“fs.s3a.enableServerSideEncryption”, “true”) sc.hadoopConfiguration.set(“fs.s3a.server-side-encryption.key”, “---------”)

using Hadoop 2.7 and spark 1.6 .


I am unable to write the data frame to S3 bucket and throwing access denied

Please let me know any thing need to be added.


I think those settings are fine. Make sure configuration values are correct to write the data.