How to set executors for static allocation in Spark Yarn

Question

I want to increase the number of executors assigned for static allocation of resources in Spark Application. Can someone tell me what code to use to do it?

Answer 1

Open Spark shell and run the following command:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.executor.instances=<number of executors>