CDN & Edge Caching at Scale

Move bytes closer, and shield the origin

A CDN exists to do two things: move bytes physically closer to users so latency drops, and absorb read traffic so your origin never sees the full load. A user in Sydney fetching from a single us-east-1 origin pays roughly 150 to 250 ms of round-trip time per request; an edge PoP 20 ms away turns that into a snappy response and, because the object is cached, the origin never handles the request at all.

There are two CDN fill models. A pull CDN is lazy: the edge fetches from origin on the first miss for an object, caches it, and serves subsequent hits locally. A push CDN is eager: you publish objects into the CDN ahead of demand. Pull is the default for almost everything because it is self-managing; push is reserved for large predictable launches (a game patch, a video premiere).

The multi-tier hierarchy and the origin shield

The structure that actually protects a fragile origin: many L1 edge caches close to users, a smaller set of L2 regional PoPs behind them, and a single origin shield in front of the origin. The shield is the key trick. When a popular object expires, thousands of edges could each miss and hammer the origin simultaneously. The shield coalesces those misses: it lets one request through to origin, holds the others, and fans the single response back out. On a burst the origin sees thousands of QPS instead of millions. Set stale-while-revalidate so the edge keeps serving the slightly stale object while one background fetch refreshes it.

Pull CDN hierarchy with an origin shield

Step 1 / 3

Stage 1 of 3: Without a CDN, a Sydney user pays roughly 150 to 250 ms of round-trip time to a single us-east-1 origin, and the origin handles every request itself.

Invalidation and cache keys

You have three tools. TTL expiry is simplest but coarse. Explicit purge is precise but slow to propagate globally and easy to over-use. The production default is versioned or content-hashed URLs: app.4f9c2a.js instead of app.js. A new deploy is a new URL, so you can cache the old one forever (immutable) and never purge; the HTML that references it gets a short TTL. This sidesteps invalidation almost entirely.

Cache-key normalization decides your hit rate. By default the key is the full URL including query string, so ?utm_source=twitter and ?utm_source=email are two cache entries for one image. Strip tracking params, normalize casing, and only Vary on headers that actually change the body (like Accept-Encoding). Vary on Cookie and your hit rate collapses to near zero.

Interview nuance: the sharpest question is "what can you cache and what must you never cache?" Static assets and public semi-dynamic HTML: yes, with micro-caching (a 1 to 5 second TTL on the homepage still collapses a 100k-QPS spike to ~20 origin fetches/sec). Personalized or authenticated responses: never at a shared edge, or you leak one user's account page to another. Do personalization with edge compute (Cloudflare Workers, Lambda@Edge) that assembles a cached shell plus a small per-user fragment.

Recap: use a pull CDN with an L1/L2/shield hierarchy so the shield coalesces misses down to ~1 fetch per object, prefer versioned URLs over purging, normalize cache keys, micro-cache semi-dynamic HTML with stale-while-revalidate, and never cache authenticated bodies at a shared edge.