How to convert JSON file to AVRO file and vise versa

Question

Hi All,

I want to convert my json file to avro and vice versa but facing some difficulty. Can anyone please help me debugging this?

val df = spark.read.json(“path to the file”)
df.write.format(“com.databricks.spark.avro”).save(“destination location”)

But I am getting the below error:

org.apache.spark.sql.AnalysisException: Failed to find data source: com.databricks.spark.avro.

Answer 1

Try including the package while starting the spark-shell:

spark-shell --packages com.databricks:spark-avro_2.10:2.0.1

And then import:

import com.databricks.spark.avro._