Apache Spark 1.6 - Transform, Stage and Store - Row level transformations – String Manipulation

Let us look into the details for performing row level transformations

  • Understand string manipulation using Scala
  • map
  • flatMap

String Manipulation

Understanding string manipulation APIs helps us processing data as part of lambda or anonymous functions used in Spark APIs

  • Extracting data – split and get required fields
  • Converting data types – type cast functions
  • Discarding unnecessary columns
  • Derive new expressions with data from different fields

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster