How to change scheduling mode in Spark

Question

The scheduling now is happening based in FIFO. I am running a multi-user service and so I want to change the scheduling mode to FAIR mode. How can I do this?

Answer 1

You can change the scheduling mode as follows:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.scheduler.mode=FAIR