Load Balancer
A component that distributes incoming traffic across multiple servers using health checks and balancing algorithms, operating at L4 (transport) or L7 (application).
By Ratish Jain · Updated
Definition
A load balancer sits in front of a pool of servers and spreads incoming requests across them. It presents a single virtual endpoint to clients while transparently routing each request to a healthy backend, improving both scalability and availability.
Why It Exists
A single server has finite capacity and is a single point of failure. Once you scale horizontally by adding more servers, something must decide which server handles each request and must stop sending traffic to a failed one. The load balancer solves both: it multiplies capacity and hides failures from users.
How It Works
An L4 balancer routes on TCP/UDP info (IP and port) — fast, protocol-agnostic, but blind to content. An L7 balancer parses HTTP, so it can route by path, header, or cookie and terminate TLS. It picks a backend using an algorithm: round-robin (rotate evenly), least connections (send to the least busy), weighted (favor bigger boxes), or IP hash (sticky by client). Continuous health checks probe each backend and pull failing ones out of rotation until they recover.
What It's Used For
- Distributing web/API traffic across a server fleet
- Removing failed instances via health checks (high availability)
- TLS termination and offloading at L7
- Blue-green and canary deployments by shifting traffic
- Session stickiness for stateful apps
Spreading Load
Tradeoffs
- Enables horizontal scaling and eliminates single points of failure
- Health checks give automatic failover
- L7 unlocks content-based routing and TLS offload
- Smooths uneven load across backends
- The LB itself can become a bottleneck or SPOF (needs redundancy)
- L7 inspection adds latency versus L4
- Sticky sessions complicate scaling and rebalancing
- Adds a network hop and operational config
When to Use / Avoid
- Use when
- You run more than one instance of a service and need to spread traffic and survive instance failures.
- Avoid when
- A single small service with no HA requirement — the extra hop and config aren't worth it yet.
- Real-world
- AWS ELB/ALB, NGINX, HAProxy, and Envoy front nearly every large-scale web service.
Interview Questions
When you need content-aware routing (by URL path, host, or header), TLS termination, or request-level features. L4 is chosen for raw throughput and non-HTTP protocols.
Run it redundantly — active-passive with a floating IP/VRRP, or multiple LB nodes behind DNS or an upstream L4 balancer, so failure of one doesn't take down the endpoint.
- →Distributes traffic across a pool behind one virtual endpoint.
- →L4 routes on IP/port; L7 parses HTTP for content-based routing.
- →Algorithms: round-robin, least-connections, weighted, IP hash.
- →Health checks pull failed backends out of rotation.
- →Must itself be made redundant to avoid a new SPOF.
Related
Everything on Archprimer is connected — follow the graph.