Question

Not able to read the file from hdfs location using spark. But getting the error after running this line

val ordersRDD=sc.textFile("/user/reshma/sqoop_import/retail_db/orders ")

ERROR:

org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/reshma/sqoop_import/retail_db/orders

Answer 1

You have to mention the hdfs path, but you are missing it out. 

For your case, you might change the command to the below:

val RDD = sc,textFile("hdfs:///user/reshma/sqoop_import/retail_db/orders")

Answer 2

Please make sure you connect to spark2-shell instead of spark-shell. 

>spark2-shell 

Once you get spark shell. The following code should work -

val ordersRDD = sc.textFile("/user/reshma/sqoop_import/retail_db/orders.csv")

ordersRDD.collect()