error Caused by org apache spark SparkException Failed to execute user defined function

Question

Hi Guys,

I installed spark in my system and tried to create one user defined function in scala. But it shows me the below error.

error: Caused by: org.apache.spark.SparkException: Failed to execute user defined function at org.apache.spark.sql.catalyst.expressions.GeneratedClass$Gen‌​eratedIterator.proce‌​ssNext.

How can I solve this error?

Answer 1

Hi@akhtar,

I think you got this error due to version mismatch of spark-mllib library with spark-core library and spark-sql library. They all should be of same version. So check these library and their versions.

Hope this will give you some idea.