CDNs & Caching
A load balancer spreads work across a pool, but every backend still sits in one place. If that place is a data center in Virginia, a user in Sydney pays for the round trip - light through fiber is fast, but 16,000 km each way is not free. A Content Delivery Network fixes this by moving copies of your content close to the user instead of moving the user's request across the planet.
Latency is distance
The dominant cost for a far-away user is not bandwidth, it is the round-trip time set by physical distance. Every request, every TLS handshake, every asset pays that toll again. A CDN operates hundreds of edge PoPs (points of presence) spread across cities worldwide. The user connects to the nearest PoP - a few milliseconds away - instead of the origin thousands of kilometers off.
Cache hit versus cache miss
An edge PoP holds a cache of your content. What happens next depends on whether the requested item is already there:
| Cache hit | Cache miss | |
|---|---|---|
| At the edge | Item is present and fresh | Item absent or expired |
| What happens | Edge serves it immediately | Edge fetches from origin, then stores a copy |
| Distance paid | User to nearest PoP only | User to PoP, plus PoP to origin (once) |
| Latency | Low - a few ms | High for this request, low for the next |
| Origin load | None | One fetch, then future hits are absorbed |
The first user in a region to ask for something triggers a miss and a fetch from the origin; the copy the edge stores turns every later request in that region into a fast hit. A busy CDN absorbs the overwhelming majority of traffic at the edge, so the origin sees only a trickle.
Say a PoP serves a popular product image with a 1-hour TTL, and over that hour it receives 10,000 requests for it from users in the region. The very first request is a guaranteed miss - the edge has never seen this object, so it fetches from origin and stores a copy. Every one of the remaining 9,999 requests, arriving before the TTL expires, is served straight from the edge: a hit.
- Cache hit ratio = 9,999 / 10,000 = 99.99%.
- Origin requests for that hour = 1, instead of 10,000.
- If the origin round trip costs 120 ms and the edge hit costs 5 ms, the average latency across all 10,000 requests works out to roughly ((1 x 120) + (9,999 x 5)) / 10,000 ~= 5.01 ms - essentially the edge latency, because the one slow fetch is amortized across thousands of fast hits.
Shorten the TTL to 60 seconds instead of an hour and, assuming requests arrive at a steady rate, you now get a fresh miss roughly every minute - 60 misses across that same hour instead of 1 - dropping the hit ratio to about 99.4% and multiplying origin load by 60x. The hit ratio is directly a function of how long the TTL is relative to how often the object is requested.
Cache hierarchy
Edges are not always alone. Many CDNs stack a tier of regional or "shield" caches between the edge PoPs and the origin. An edge miss checks the regional cache before troubling the origin, so a miss in one city can still be a hit regionally. This keeps origin fetches rare even when many edges warm up at once, and shields the origin from a stampede of simultaneous misses.
TTL and invalidation
Cached content cannot live forever, or updates would never reach users. Two mechanisms bound its lifetime:
- TTL (time to live) - the origin tags a response with how long it may be
cached (via
Cache-Control: max-age). When the TTL expires the edge treats the item as stale and revalidates or refetches on the next request. - Invalidation / purge - an explicit command telling PoPs to drop an item now, before its TTL runs out. Used when you ship a fix and cannot wait for the clock - though a global purge is slower and costlier than letting a short TTL expire.
The everyday pattern is a long TTL plus versioned filenames: app.9f3c1.js. New
content gets a new name, so it is a new cache key and there is nothing stale to
purge.
If a response is cached with Cache-Control: max-age=86400 (24 hours) and, an
hour later, the underlying content is taken down for a legal reason, replaced
with corrected data, or found to contain a bug or leaked secret, every edge
that already cached it keeps serving the old bytes for the remaining 23
hours - the CDN has no way to know the origin changed its mind unless it is
told. This is exactly why "unpublish" and "hotfix" workflows cannot rely on
TTL expiry alone: they need an explicit purge/invalidation call issued the
moment the change happens, fanned out to every PoP that might hold a copy.
Even then, purges are not instantaneous across a global network, so there is
a real window - typically seconds to a couple of minutes depending on the
CDN - where some edges have purged and others have not, meaning two users
hitting different PoPs can briefly see different versions of the same URL.
The versioned-filename pattern sidesteps this entire problem for static
assets (a new file name is trivially a cache miss everywhere), which is why
it is preferred over purging whenever the asset can be renamed.
What to cache, and what not to
CDNs shine on content that is the same for everyone and changes rarely:
- Cache - static assets: images, CSS, JavaScript bundles, fonts, videos, downloads. Identical for every user, so one cached copy serves thousands.
- Do not cache (or cache carefully) - dynamic, per-user, or sensitive responses: a logged-in dashboard, a shopping cart, a bank balance. Serving one user's cached page to another would be wrong and a privacy leak.
Static assets are the sweet spot because they are immutable and shared - the same bytes for everyone. A cached hit means the origin never runs code, touches a database, or pays the long-distance round trip. That is the biggest, cheapest win a CDN offers, which is why versioned static files are cached aggressively while dynamic HTML usually is not.
CDNs steer users with DNS
How does a user reach the nearest PoP if everyone uses the same hostname? The
CDN answers through DNS. When the client resolves cdn.example.com, the CDN's
name servers return the IP of a PoP close to the resolver's location, so the
same name leads different users to different edges. If you have not yet seen how
name resolution works, review
DNS Resolution first - it is the
mechanism a CDN leans on to place every user at the right edge.
Recap
- Latency is dominated by distance; CDNs cut it by caching content at edge PoPs near users.
- A hit serves from the edge instantly; a miss fetches from origin once, then future requests hit.
- A cache hierarchy shields the origin; TTL and purges bound how long copies live.
- Cache static, shared assets aggressively; keep dynamic per-user responses off the cache.
- DNS is what routes each user to the closest PoP.