How to make Spark skip node locality

Question

I have setup cluster with rack information and I want the application to look for rack locality without looking for node locality. How can I configure this?

Answer 1

If you set the node wait time to 0, then the application will directly look in the rack locality. Try this:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.locality.wait.node=0