Trying to execute this statement

val dataset = Seq((0, “hello”),(1, “world”)).toDF(“id”,“text”)
val upper: String => String =_.toUpperCase
import org.apache.spark.sql.functions.udf
val upperUDF = udf(upper)
dataset.withColumn(“upper”, upperUDF('text)).show

scala> spark.udf.register(“myUpper”,(input:String) => input.toUpperCase)
:27: error: not found: value spark
spark.udf.register(“myUpper”,(input:String) => input.toUpperCase)
^

sparksql

any suggestions what i am missing here ? i tried to execute the statements as it is .

Its should be sqlContext.udf.register(“myUpper”,(input:String) => input.toUpperCase)
“spark” doesn’t have any package with name udf.

1 Like