Hi, I want to write a custom cassandra codec and i want to use that codec in my spark application while reading the data from cassandra table .
Basically the custom codecs are used to convert one column type to another while reading from cassandra. for example i have a timestamp column in cassandra table ,but i want to read it as long and save as parquet file to HDFS , then this cutom codecs will be usefull instead of doing an additional transformation in the spark code.
I want to know how to set/register this custom cassandra codecs in spark ?
I had tried with solution provided in the below link but it does not work , any ideas around this will be helpfull.
Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs
- Click here for access to state of the art 13 node Hadoop and Spark Cluster