How to create paired RDD using subString method in Spark

Question

I have a file with and ID and some values then how to create a paired RDD using subString method in Spark?

Answer 1

Hi,

If you have a file with id and some value, then you can create paired rdd with id as key and value as other details:

Here is an example of doing that below:

val pRDD2[(Int),(String)]=sc.textFile(“path_of_your_file”)
.keyBy(line=>line.subString(1,5).trim().toInt)
.mapValues(line=>line.subString(10,30).trim())