47432/difference-between-rdd-as-val-and-var
RDD is considered immutable. I tried to create an RDD with val and var like given below. I can see I was able to change RDD definition created using var. If it"s immutable why was I able to use var to create an RDD?
scala> var df = sc.textFile("/user/saifu/problem5/text") df: org.apache.spark.rdd.RDD[String] = /user/saifu/problem5/text MapPartitionsRDD[13] at textFile at :27 scala> val db = sc.textFile("/user/saifu/problem5/text") db: org.apache.spark.rdd.RDD[String] = /user/saifu/problem5/text MapPartitionsRDD[15] at textFile at :27 scala> df = sc.textFile("/user/sarfu/problem5/text-uncompress") df: org.apache.spark.rdd.RDD[String] = /user/sarfu/problem5/text-uncompress MapPartitionsRDD[17] at textFile at :29
Variable declaration can be done in two ways
1. val --> immutable variable
Refer to the screenshot below:
2. var --> mutable variable
I hope this helps.
I am pretty sure createOrReplaceTempView just replaced ...READ MORE
Both 'filter' and 'where' in Spark SQL ...READ MORE
Hi, persist () allows the user to specify ...READ MORE
Please go through the below explanation : Full ...READ MORE
Instead of spliting on '\n'. You should ...READ MORE
Firstly you need to understand the concept ...READ MORE
org.apache.hadoop.mapred is the Old API org.apache.hadoop.mapreduce is the ...READ MORE
Hi, You can create one directory in HDFS ...READ MORE
Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE
Hey, When we try to compare two instances ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.