Load Balancing: L4 vs L7 & Health Checks

Scale out, not up, and notice when a node dies

A load balancer is the primitive that lets you scale out instead of up. Once one machine cannot serve your traffic, you run N identical machines and put a load balancer in front to spread requests across them. Everything else in this lesson is about doing that spreading correctly and about noticing when one of those N machines is dead.

The layer

An L4 (transport) load balancer works at the TCP/UDP level. It sees IP addresses and ports, not HTTP. It picks a backend, forwards packets, and stays out of the way. Because it never parses the request body or terminates TLS, it is extremely fast and cheap per connection, and it is content-blind: it cannot route /api/* to one pool and /images/* to another. AWS NLB and IPVS are L4. An L7 (application) load balancer terminates the connection, reads the HTTP request, and can route on host, path, header, or cookie. It usually terminates TLS, can retry failed requests, inject headers, and do sticky routing. AWS ALB, Nginx, Envoy, and HAProxy in HTTP mode are L7. The cost is CPU and latency: it does real work per request. Rule of thumb: use L7 when you need HTTP-aware routing, TLS termination, or per-request features, and L4 when you need raw throughput or non-HTTP protocols.

Interview nuance: Interviewers love to ask "L4 or L7 and why," then probe TLS. The clean answer: L7 terminates TLS at the edge so backends speak plain HTTP inside the trusted network (or re-encrypt for zero-trust); L4 passes TLS straight through, so the backend does the handshake and the LB never sees plaintext.

Where TLS ends is the whole difference, and every other row follows from it:

Table

Read the first row and the rest is derivable. Every L7 capability follows from decrypting at the edge, and every L4 limitation follows from not decrypting. Answer the TLS question and the L4-or-L7 answer falls out of it.

Property	L4 (transport)	L7 (application)
TLS	passes straight through; the LB never sees plaintext	terminates at the edge
Can see	IP and port only	host, path, headers, cookies, body
Route /api/* to its own pool	no	yes
Retries, sticky sessions, header injection	no	yes
Cost per request	minimal; it forwards packets	real CPU; it parses every request
Protocols	any TCP or UDP	HTTP and HTTPS
Examples	AWS NLB, IPVS	AWS ALB, Nginx, Envoy, HAProxy

The algorithm

Round robin rotates evenly and is fine when every request costs about the same. Least connections sends the next request to the backend with the fewest in-flight connections, which is the right default when request durations vary a lot (some calls take 2 ms, some take 2 s), because it naturally avoids piling long requests onto one node. Weighted variants let a bigger box take more traffic. Consistent hashing pins a given key (user id, cache key) to the same backend so you get cache affinity with minimal reshuffling when the pool changes.

Failure detection and draining

Active health checks have the LB probe each backend on a schedule (GET /healthz every few seconds); miss a threshold of probes and the node is marked down and pulled from rotation. Passive health checks watch real traffic: if a backend returns errors or times out, eject it. You want both. When you deploy, you do not want to kill in-flight requests, so you use connection draining: the LB stops sending new requests to a node, waits for existing ones to finish (up to a timeout), then removes it. Pair that with graceful shutdown in the app (stop accepting, finish work, exit).

Two traps. First, prefer stateless services so any node can serve any request; sticky sessions (pinning a user to one node) are a crutch that breaks when that node dies and complicates deploys. Push session state to Redis instead. Second, the load balancer itself is a single point of failure. One LB in front of ten app servers just moves the SPOF up a layer. Run it redundant: active-active pairs, or an anycast VIP fronting multiple LBs, with health-checked failover.

        anycast VIP (redundant LBs)
              |
        [ L7 load balancer ]  <- TLS terminate, path routing, least-conn
         /        |        \
     app-1     app-2     app-3   (stateless; session in Redis)
       ^ active healthz probes every 3s; drain on deploy

Recap: Pick L4 for raw speed or L7 for HTTP-aware routing and TLS, use least-connections when durations vary, combine active and passive health checks with connection draining, keep services stateless, and never leave the LB itself un-replicated.