How does horizontal scaling decrease load of a system

Question

say I have a 3 node LXD cluster running under a load balancer. Traffic begins ramping up and I need to scale horizontally by adding another node into the cluster. At this point, I have a 4-node cluster under the LB.

In what ways in this approach beneficial? I'm trying to understand if it reduces the load of the overall baremetal that contains the nodes or just allows for more requests to process. In the case that it's more requests that are allowed to process through, would I want to me measuring the load average of each individual container?

Answer 1

If there are spare CPU cycles and sufficient network bandwidth, adding a new node will always allow more requests to be handled simultaneously. In some cases though this will reduce the responsiveness of each request.

If CPU load is high then adding another node (on the same box) will reduce throughput (extend response time -- because you're asking the cpu to do more work that it can do simultaneously). If instead the load is high due blocking IO then adding another node on the box should not significantly affect the processing time for each request.

In the blocking IO case you can add new nodes until CPU is at its threshold level -- you never want to max out CPU, you want to have the thresholds at, maybe, 75%, to allow for variations in load.