How to register cassandra custom codecs in spark?

apache-spark

#1

Hi, I want to write a custom cassandra codec and i want to use that codec in my spark application while reading the data from cassandra table .

Basically the custom codecs are used to convert one column type to another while reading from cassandra. for example i have a timestamp column in cassandra table ,but i want to read it as long and save as parquet file to HDFS , then this cutom codecs will be usefull instead of doing an additional transformation in the spark code.

I want to know how to set/register this custom cassandra codecs in spark ?

I had tried with solution provided in the below link but it does not work , any ideas around this will be helpfull.


Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster