Load Balancing
Published on
Load balancing distributes requests or connections across eligible backends. It can improve capacity and availability, but it also becomes part of the request path and must have its own redundancy, capacity, health, and observability design.
Layers and decisions
- DNS and global traffic management: choose a region or edge using policy and health, subject to caching and propagation delay.
- Layer 4 balancing: route connections using IP and port with limited application awareness.
- Layer 7 balancing: inspect HTTP or another application protocol for host, path, header, identity, or content routing.
- Client-side balancing: let a client choose from discovered endpoints, shifting policy and failure handling into clients.
Algorithms include round robin, least connections, weighted distribution, hashing, and latency-aware selection. The correct choice depends on request cost, connection duration, cache locality, and backend capacity.
Health and failure
Use readiness to remove a backend before sending traffic and graceful draining to protect in-flight work. A shallow health endpoint may report success while a critical dependency is broken; a deep check can remove every backend during a shared dependency failure. Choose checks based on the failure you want routing to contain.
Retries need budgets and deadlines because a balancer can multiply load during failure. Track request rate, errors, duration, active connections, rejected work, backend distribution, retry volume, and saturation.
Sticky sessions can preserve locality but create uneven load and complicated failover. Prefer externally managed session state when the user contract permits it.