Can the executor core be greater than the total number of spark tasks

Question

What happens when the executor core be greater than the number of spark tasks? Is this scenario possible? If yes what happens in extra core.

Answer 1

Hi@Rishi,

Yes, it is possible. If executor no. is more than tasks, in that scenario each task gets one executor core. Extra cores are allocated to executors that have no active tasks. They sit ideal. But these executors consume your memory and slow down the process.