How to parse a textFile to csv in pyspark

Question

import pyspark

from pyspark import SparkContext

sc=SparkContext()

data=sc.textFile("/user/edureka_968359/appleStore/appleStore.csv")

I have read a csv file as a textfile and now want to parse it to csv

Answer 1

Hi,

Use this below given code, it will help you.

import pyspark
from pyspark import SparkContext
from pyspark import SparkConf
sc = SparkContext.getOrCreate(SparkConf())
df = sc.textFile('file:/root/read.csv')
df.saveAsTextFile('/root/name.csv')

After this check your output, it will be in csv format.

Hope this helps!

To know more about Pyspark, it's recommended that you join PySpark course online.

Thanks.