Scalability and Elasticity
Published on
Scalability is the ability to handle growth while meeting explicit latency, throughput, reliability, and cost objectives. Elasticity is the ability to add and remove capacity in response to demand. Neither requires microservices, and neither guarantees availability.
Model demand and capacity
Estimate request rate, concurrency, payload size, storage growth, fan-out, cache hit rate, background work, and peak shape. Identify the constrained resource and how quickly additional capacity becomes useful.
- Vertical scaling adds resources to one node. It can be simple and efficient but encounters hardware, restart, and failure-domain limits.
- Horizontal scaling adds nodes or partitions. It requires workload distribution, stateless or explicitly partitioned state, consistency decisions, and operational coordination.
Optimize after measuring. Common constraints include serialized work, connection pools, hot partitions, synchronous fan-out, lock contention, queue consumers, external quotas, and downstream capacity. Caching and asynchronous processing can move or defer load, but both introduce freshness, backlog, and failure behavior.
Test the system
Use representative traffic, data size, cache state, dependencies, and failure conditions. Measure latency percentiles, throughput, errors, saturation, queue growth, retry amplification, and cost. Determine the sustainable limit and what happens beyond it.
Design overload behavior before autoscaling: bounded queues, deadlines, admission control, load shedding, and degraded responses protect existing work while new capacity starts.