System Design Simple logoSystem Design Simple Follow on YouTube

Free courseModule 6: Caching

Lesson 37 of 92: Caching layers

Course outline

System design caching: five layers that can keep an old price

System design caching can answer reads at five layers. Our seeded demo sent 215 of 1,000 reads to the database and tracked old prices after a write.

In this lesson you will learn

  • Which five caches can answer a read before the database
  • Why a write can clear two layers and leave three holding old data
  • What the Age header does between caches, and what happens without it
  • How to budget staleness for the whole chain

Builds on Lesson 24: Wide-column stores 6 min read Next: Security monitoring: login anomaly detection

System design caching: five layers that can keep an old price: diagrams from the video

Drafted with AI assistance, then checked and edited by a human engineer · How we write

System design caching can answer a read at five layers before it reaches the database: the browser, CDN edge, web server, application and database-side cache. In a seeded 1,000-read demo, only 215 reads reached the database, and after one price change the CDN still held an unexpired old-price entry 114.2 seconds later.

Where can system design caching answer a read?

Follow one product page read from a phone to the database. Each stop can keep a copy and answer the next request itself.

LayerWho it servesWho can clear itUsual freshness rule
Browser or app cacheone useronly that deviceCache-Control: max-age
CDN edgeeveryone near that edgethe CDN's purge APIs-maxage, else max-age
Web server or reverse proxyeveryone behind ityour ops toolingresponse headers, else its config (nginx proxy_cache_valid)
Application cacheone app instancethe app itselfa TTL plus delete on write
Database-side cacheevery app instancethe app itselfa TTL plus delete on write

Browser, CDN and reverse-proxy caches follow HTTP caching rules (RFC 9111), subject to their configuration. max-age applies to every cache, and s-maxage overrides it for shared caches such as a CDN (MDN Cache-Control). A web server cache adds its own settings: in nginx, proxy_cache_valid sets a lifetime, and caching headers in the response take priority over it unless you tell nginx to ignore them. The last two are code you own, usually a map in process memory and a shared store such as Redis or Memcached.

The CDN layer has its own trade-offs, push or pull, covered in CDN push vs pull. DNS answers are cached too, one step before any of these, as shown in what happens when you type google.com.

What the demo ran

labs/sd-37/measure-five-cache-layers.mjs is one Node script with a virtual clock and a fixed seed. It builds all five layers in one process: 40 clients each with a private cache, 3 CDN edges, 4 web servers with a response cache, 4 app instances with an in-memory map, one shared database-side cache, and a SQLite items table with 200 rows in node:sqlite. A load balancer spreads requests over the origin servers, as in load balancing algorithms.

The freshness rules are max-age=60 on clients, s-maxage=120 at the CDN, 30 s in the web cache, and 60 s and 120 s in the app and database-side caches. The app deletes its own entries and the database-side entry when it writes.

Latency in this demo is modeled. Each read adds up assumed hop costs (25 ms phone to edge, 70 ms edge to origin, 1 ms app to database, 4 ms for a disk query) plus seeded jitter. The lab sends no real network traffic, so these numbers show only the shape of the effect.

Terminal output of the sd-37 demo: who answered 1,000 reads, modeled medians, and how long each layer held the old value
The printed output of node labs/sd-37/measure-five-cache-layers.mjs, run on 2026-11-14. Every number in this article comes from it.

Who answered the 1,000 reads?

Reads pick items with a Zipf skew, so item 1 is the most popular and a long tail is rarely read. Over 1,000 reads:

Answered byReadsModeled median
Browser cache790.05 ms
CDN edge54225.26 ms
Web server cache1396.19 ms
Application cache1598.73 ms
Database-side cache13697.94 ms
Database (SQLite)215103.08 ms
caches answered: 79 + 542 + 13 + 15 + 136 = 785
reached the database: 1,000 - 785 = 215  (21.5%)
last 250 reads, caches warm: 36 of 250  (14.4%)

The edge did most of the work because it sits in front of everything and is shared by every client that uses it. The three origin caches answered only 164 reads between them: by the time a read gets past the edge, the popular items have already been served.

A grid of 1,000 cells, one per read, filled by which cache answered: 79 browser, 542 CDN edge
One cell per read. The CDN edge alone answered 542 of 1,000.

What happens when the price changes?

At t = 250 s the script raises the price of item 1, the hottest row, from $10.01 to $11.01 in SQLite. The app deletes item 1 from its own maps and from the database-side cache, because it owns both. Then the same traffic runs for 400 more seconds, and every 0.1 s the script records which layers still hold an unexpired old entry.

LayerLast sampled with the old value (timers only)With a CDN and web purge
Browser cache+54.2 s+54.2 s
CDN edge+114.2 s0
Web server cache+24.2 s0
Application cache00
Database-side cache00

The table counts entries each layer still held. Responses that reached users are counted separately: with timers only, 27 carried the old price, the last at +112 s. Purging the CDN and the web cache at the moment of the write cut that to 2 responses, the last at +36 s. These sampled times come from one run and are not upper bounds; the budget section below works out the worst case.

Why a purge cannot reach the browser

A server cannot push a delete into a browser cache. The browser hears from you only when it sends a request, and with max-age=60 it has no reason to send one for up to 60 seconds. In the purge run, one client's copy was still fresh 54.2 s after the change.

Clear-Site-Data does not change that, because it arrives on a response the browser asked for. What works is set before the change: a short max-age for data that changes, or no-cache so the browser revalidates every time. Versioned URLs also work, because a new value lives at a new address the browser has never cached.

What the Age header does between caches

A cache serving a stored response without validation must send its current age in Age (RFC 9111 section 4, MDN Age). The next cache calculates current age using the received age, response timing and time stored locally, then compares it with the applicable freshness lifetime (section 4.2.3). In the demo, the CDN passes its age to clients, and each client counts its 60 seconds down from that age.

This model's web cache does not send Age to the CDN, and the model treats a missing Age as zero. So the CDN stores a copy that may already be up to 30 seconds old in the web cache and gives it a new 120 seconds. Real caches can also estimate age from the Date header and response timing (RFC 9111 section 4.2.3), so a missing Age does not always restart the clock, but check that every cache in your chain sends it.

How to budget staleness for the whole chain

Write the worst case down per path, from the write to the last cache that could still serve the old value. These bounds assume successful invalidation of every instance, no in-flight old read refilling a cache after the write, no revalidation that renews an old representation, and no stale serving.

HTTP caches keep the age, no stale serving:  about the largest lifetime
  CDN s-maxage 120 s; browser max-age 60 s counted from the CDN's Age   -> about 120 s
age reset at the web cache:  lifetimes add up
  web 30 s + CDN 120 s                                                   -> about 150 s
purge CDN and web on write:  only the browser remains
  browser max-age 60 s                                                   -> about 60 s

The app and database-side caches need their own line in the budget: Age says nothing about them, because a freshly generated response can carry old data read from either. Here the app deletes both on write, so they add 0 s. Then decide which layers you purge on write, which you shorten, and which data should not be cached at a shared layer at all. A price is a good candidate for a short browser lifetime and a CDN purge; a product photo at a versioned URL can be cached for a year. The same thinking about a shared, delayed copy appears in the Instagram feed design.

What we saw when we ran it

We ran node labs/sd-37/measure-five-cache-layers.mjs on a Mac on 2026-11-14, and the whole run took 0.03 s. The script prints a table of which layer answered each read, then a second table of how long each layer kept the old price.

Terminal output of our sd-37 run: node measure-five-cache-layers.mjs Our terminal after the sd-37 run, with both tables and the RESULT line.

In the first table, the CDN edge answered 542 of 1000 reads, and 215 reached SQLite, falling to 36 of the last 250 once the caches were warm. The latencies in that table are modeled, so we read them as a rough shape only.

The second table samples every layer every 0.1 s after the update. With TTLs alone, the last sampled time the CDN edge still held an old entry was +114.2 s, against +54.2 s for the browser, and 27 responses carried the old value, the last at +112 s. With the CDN and web caches purged, 2 responses carried the old value, the last at +36 s, but a browser still held its old copy at +54.2 s, so in this model the browser lifetime set how long the old price lasted once the shared caches were purged.

Screenshot of the official page: RFC 9111: HTTP Caching (freshness, Age, Cache-Control) The official RFC 9111 page on HTTP caching.

RFC 9111 confirms that a cache serving a stored response without validation must send its current age in Age.

Limits of this demo

  • Latency is modeled from assumed hop costs, not measured. The one real number, a SQLite point lookup of about 1.4 µs on this Mac, is printed and not used.
  • The chain is simplified HTTP, with no revalidation, no stale-while-revalidate, and the purge is assumed instant on every edge.
  • The simulation is sequential; invalidation across all app instances is instant.
  • Staleness is sampled every 0.1 s, and the old-value count depends on the seed and the traffic mix.

Our take

Caching at five layers took the database from 1,000 reads to 215 in this demo, and most of that came from the shared edge. This model deletes the app and database-side entries on write; CDN and web caches need separate purges, while browser copies remain until expiry. List every cache, its TTL and how it is cleared, and write the worst-case staleness for the whole path before you choose TTLs.

The shop, prices and traffic are synthetic, made from a fixed seed, and the latencies are modeled from assumed costs. The layers are a typical design for teaching, not any company's system.

Frequently asked questions

What are the five places a cache can sit?

The browser or app on the device, a CDN edge near the user, a response cache in the web server or reverse proxy, an in-memory cache inside the application, and a shared cache in front of the database such as Redis or Memcached.

Can a server purge a browser cache?

No. The browser only hears from the server when it sends a request. Short max-age, no-cache with validation, or versioned URLs are the tools; Clear-Site-Data only works on a response the browser actually asks for.

What does the Age header do?

Age reports how old a stored response already is. Downstream caches include that age when calculating remaining freshness; a copied response does not automatically receive a new full lifetime.

Sources

  1. RFC 9111: HTTP Caching (freshness, Age, Cache-Control)rfc-editor.org
  2. MDN: Cache-Control headerdeveloper.mozilla.org
  3. MDN: Age headerdeveloper.mozilla.org
  4. MDN: Clear-Site-Data headerdeveloper.mozilla.org
  5. nginx: ngx_http_proxy_module (proxy_cache, proxy_cache_valid)nginx.org
  6. Redis docs: client-side cachingredis.io

cachingsystem designcdnhttp cachingcache-controlage header

System Design Simple logo
System Design Simple

Written by a software engineer and system architect with 10+ years of building production web, payment, ticketing and accounting systems in Japan and Nepal. About us