CDN push vs pull: why Netflix doesn't buffer in Kathmandu

Every byte a viewer receives has to travel from wherever it is stored, so a video kept only in one faraway data center pays a long round trip on every chunk, and a player with a small buffer stalls. A content delivery network moves copies close to viewers, but CDN push vs pull, whether copies are pulled on demand or pushed ahead of time, decides whether the very first viewer of new content still waits. The question to be able to answer: why does a streaming service copy video near you before you press play?

Prerequisites

Round trip time and why distance wins. A round trip (RTT) is the time for a small request to reach a server and the reply to come back. Light in fiber covers roughly 200 km per millisecond, and real routes are longer than straight lines, so a server on another continent costs hundreds of milliseconds per round trip no matter how fast it is. In the previous part on DNS we saw that a lookup is a sum of round trips; fetching content is the same: every request that must cross an ocean pays the ocean.

Latency is not bandwidth. A fat pipe moves many bytes per second but does not make the first byte arrive sooner. A player needs bandwidth for picture quality and latency for start time; the article on latency and throughput covers the difference. This one is mostly about latency.

Caching. A cache keeps a copy of something close to whoever needs it, with a rule for when the copy is too old. Read replicas in one database writes, ten read are caches of a database; DNS resolvers cache names; a CDN caches files and video.

Definitions

How it works: a request through an edge

A player does not download one huge file. It fetches a manifest (an HLS playlist, defined in RFC 8216, or a DASH MPD) that lists the segments, then fetches segment 1, segment 2 and so on, each as a separate HTTP GET. Each segment is just a URL with bytes behind it, so any HTTP cache can store it; that is what makes CDNs work for video.

Here is a single segment request through a pull edge:

1. Routing. DNS or anycast steers the player to a nearby edge; behind that edge, a load balancer or reverse proxy often spreads requests over several cache servers.
2. Cache key. The edge builds a key from the request, usually the host and path, sometimes selected headers or query parameters.
3. Lookup. If the key is in the cache and the copy is fresh, it is a hit: the edge streams the bytes back. Cost: one short round trip plus a little server time.
4. Miss. If the key is absent, the edge opens (or reuses) a connection to the origin, asks for the object, and waits for the first byte to cross the long path.
5. Fill. As the bytes arrive the edge writes them to its cache (memory or disk) and forwards them to the player. Writing, indexing and possibly evicting another object take time; our lab calls this the miss overhead.
6. Reuse. The next viewer who asks for the same key gets a hit.

Notice what a miss costs. It is not just "the file is far away". The viewer pays the short hop to the edge, then the edge pays the long hop to the origin, plus the origin's own server time, plus the edge's lookup and store work. So for that one request, a pull edge can be slightly slower than going straight to the origin. The benefit only appears on the second request for the same object.

What the player does with those times

A player keeps a buffer of downloaded but unplayed media. A stall (rebuffering) happens when that buffer drains to zero. In general the buffer level B changes as

dB/dt = (delivery rate / media bitrate) − 1      (seconds of media gained per second, while playing)
stall when B reaches 0

So stalls depend on how many seconds of media are buffered and on the sustained delivery rate relative to the bitrate, not on one request's latency. Our lab uses a deliberately simplified version: the player asks for each segment a fixed lead time before it is due, and

lab rule:  stall if   TTFB + transfer time  >  lead time (300 ms)

That makes per-request latency visible, which is the point of the model. Real players keep many seconds of buffer, so steady playback rarely stalls on a few hundred milliseconds of TTFB.

Startup is where latency shows most, but it has more parts than one segment fetch: manifest fetch, DRM licence if any, initialization segment, then media, transfer and decoder start. Our lab reports modeled startup time, assuming playback waits for the complete first segment: TTFB of segment 1 plus its transfer. A player using progressive delivery (for example low-latency modes that decode partial segments or CMAF chunks) can start before the whole segment arrives, so treat the lab number as a model of one component, not a measured start time.

CDN push vs pull: who triggers the fill?

The difference is who triggers the copy.

In a pull design the viewer's request triggers it. Nothing is stored until someone asks, so the cache holds what people actually watch, with no planning. The price is the cold miss: the first viewer of each object at each edge pays the long trip, and if many press play on a new episode at once, the origin sees a burst.

In a push design the operator triggers it. A fill job copies chosen content to the edges before it is requested, ideally when links are idle, so the first viewer is as fast as the hundredth. The price is storage and prediction: every byte pushed that nobody watches wastes disk and transfer.

Netflix publishes one relevant fact about its own CDN. Its Open Connect overview says content is placed on Open Connect appliances ahead of time (proactive caching), mostly during off-peak hours. That is push, timed for when the network is quiet. We do not describe any other internal detail of their system; everything else in this article is a typical design.

A hybrid is one option when popularity is skewed, with a small head of titles or files taking a large share of requests and a long tail getting a few each:

• Push the head. Content that is predictably popular (a new release, a launch page, a game patch) is pre-positioned at edges during a fill window.
• Pull the tail. Everything else is fetched on demand and kept while it is used, evicted when space is needed (Nginx, for example, removes least recently used data when the cache exceeds max_size).
• Prefetch within a title. A pull edge can fetch segment N+1 when segment N is requested. That only turns the next request into a hit if the edge guesses the rendition (bitrate) the player will actually pick, the prefetch completes before the player asks, there is spare origin and link capacity for it, and the object is still resident when requested. With adaptive bitrate, a wrong rendition guess is pure wasted fill.

The real question is the cheapest way to make the first request at each edge a hit for content that matters.

When does pushing pay? A break-even example

Model the requests for one object at one edge as a Poisson process with rate λ, and let T be the residency window (how long a pulled copy would stay before eviction or expiry). Assume one fill per window, and the same object size S for push and pull. Then:

P(pull fetches the object in a window) = 1 − exp(−λT)
expected pull bytes per edge per window = S × (1 − exp(−λT))
push bytes per edge per window          = S                    (pushed whether or not anyone asks)
push / pull byte ratio                  = 1 ÷ (1 − exp(−λT))

Worked numbers:

λT = 0.1  → 1 − e^−0.1 = 0.0952   → 9.52% of edges ever request it   → push moves ≈ 10.5× the pull bytes
λT = 1    → 1 − e^−1   = 0.632    → push moves ≈ 1.58×
λT = 3    → 1 − e^−3   = 0.950    → push moves ≈ 1.05×

So for a title expected to get about three requests per edge per window, pushing costs almost nothing extra in fill bytes and removes the cold miss for the first viewer at every edge. For a tail title at λT = 0.1, pushing to every edge spends about 10.5 times the transfer and storage of pulling, to save one miss penalty (in our model, 335 ms against 75 ms of modeled startup) for the roughly 9.5% of edges that see a viewer at all. Weigh the extra bytes, priced in off-peak transit and edge disk, against the value of that first-request latency. The model ignores popularity changes inside the window, partial fills and push-time failures; it is a placement heuristic, not a forecast.

How do HTTP caching rules control a CDN?

A pull edge is an HTTP shared cache, and the rules for shared caches are in RFC 9111. The origin controls the edge mostly through response headers.

Freshness. A stored response is fresh while its age is below its freshness lifetime. The lifetime comes from Cache-Control: s-maxage (for shared caches only), else max-age, else the Expires header, else a heuristic. In formula form:

fresh  if  current_age < freshness_lifetime
freshness_lifetime = s-maxage, else max-age, else (Expires - Date), else heuristic

Age. When a cache serves a stored response it sends an Age header: its estimate, in whole seconds, of the time since the response was generated or last validated at the origin. Per RFC 9111 section 4.2.3 it is computed from the upstream Age value, the apparent age (max(0, response_time − date_value)), a correction for request transit delay (response_delay), and the time the copy has sat locally:

corrected_initial_age = max(apparent_age, age_value + response_delay)
current_age           = corrected_initial_age + resident_time

Age: 0 therefore does not prove a miss: a fresh hit served within a second of filling or revalidation also shows 0. To diagnose, log the cache's own status (in Nginx, $upstream_cache_status: MISS, HIT, EXPIRED, STALE, UPDATING, REVALIDATED, BYPASS). For caches that generate RFC-compliant Age, repeated hits over several seconds should show an increasing age. Do not rely on that for the Nginx example below: it is not set up to generate a growing Age, so an absent or unchanged header there is not a cache failure. Use X-Cache-Status and the access log instead.

Revalidation. When a copy is stale, the cache does not always download it again. It can send a conditional request with If-None-Match (using the stored ETag) or If-Modified-Since. If the origin answers 304 Not Modified, only headers cross the long path and the stored body is reused.

Serving stale. The stale-while-revalidate and stale-if-error extensions (defined in RFC 5861, documented on MDN) let a cache answer with a slightly stale copy while it refreshes in the background, or while the origin is failing. Viewers get a hit-speed answer; the refresh pays the long trip without anyone waiting.

Private and no-store. private forbids shared caches from storing a response (it is meant for one user), and no-store forbids caching anywhere. Personal pages, account data and signed URLs often fall here.

For video this gives a clean split:

The trick behind the first row: change the URL, not the content. Publish a changed file under a new name (a hash or version in the path), and nothing needs purging. This avoids update-driven invalidation only: cold requests, eviction, expiry and new edges still cause fills.

Hit ratio, origin load and tiered caches

With a hit ratio h, the expected TTFB at an edge is a weighted average. The origin-load line holds only for a simple single-tier model, with no request coalescing, no revalidation (a 304 is still an origin request but carries no body), no warming and no shield:

E[TTFB] = h × TTFB_hit + (1 − h) × TTFB_miss
origin requests per second ≈ (1 − h) × edge requests per second   (single tier, no coalescing)

Plug in our model's mean values, 15 ms for a hit and 273 ms for a miss:

h = 0.90 → 0.90 × 15 + 0.10 × 273 = 13.5 + 27.3 = 40.8 ms
h = 0.99 → 0.99 × 15 + 0.01 × 273 = 14.85 + 2.73 = 17.58 ms

Going from 90% to 99% hits more than halves the average and cuts origin traffic tenfold, from 10% of requests to 1%. That is why operators chase the last few percent: the origin and the long links are the expensive, shared part, and, as in why a system fast for one user is slow for a million, misses are where queues form.

Averages hide the tail, but be precise about where it shows. In this two-point model (99% of requests at 15 ms, 1% at 273 ms), using the inverse-CDF definition (the pth percentile is the smallest x with F(x) ≥ p), the p99 is 15 ms because exactly 99% of requests are at or below it, while the p99.9 is 273 ms. Drop the hit ratio to 98.9% and the p99 jumps to 273 ms. So near the target hit ratio, p99 is unstable, and p99.9 is the percentile that reliably exposes miss cost.

Origin shield and tiered caching

If you have a thousand edges and each one pulls a new object independently, the origin can receive a thousand requests for the same bytes. An origin shield (also called a tiered or regional cache) puts one more cache layer in between: an edge miss goes to the shield, and only a shield miss goes to the origin.

miss at edge, hit at shield:  TTFB ≈ RTT(viewer→edge) + RTT(edge→shield) + server times
miss at both:                 TTFB ≈ RTT(viewer→edge) + RTT(edge→shield) + RTT(shield→origin) + server times
origin fills per object       ideally one successful fill per cache key per shield per residency period
                              (with effective request coalescing), instead of one per edge

That is an ideal, not a bound. Expiry starts a new residency period, eviction forces a refill, a failed fill (origin error or timeout) is retried, and concurrent misses that escape coalescing each go upstream. The shield adds a hop on a full miss, but it protects the origin and turns many edge misses into shorter shield hits. Our lab does not model a shield; the formula shows what it would add.

Request coalescing

Shields and edges have one more problem: a burst. When a popular object expires or first appears, many requests can miss at the same instant. Without care, each one starts its own origin fetch, a pattern called a cache stampede or thundering herd. Request coalescing (collapsed forwarding) lets the first miss fetch while the others wait for that same fetch. In Nginx this is proxy_cache_lock on; Varnish coalesces requests for the same object by default.

The Nginx lock is bounded in time, and the bounds decide how well it works. With proxy_cache_lock_timeout 5s (the default), a request that has waited 5 seconds for the lock is passed to the origin anyway, and its response is not cached. Separately, proxy_cache_lock_age 5s lets one more request go upstream if the fill in progress has not completed within 5 seconds. So a slow origin, one that takes longer than 5 s to deliver a large segment, turns a coalesced burst back into several origin requests. In the best case one origin request serves the whole burst; under a slow or failing origin, expect more, and watch upstream request counts per key rather than assuming one.

Worked example

The video's scenario: a viewer in Kathmandu presses play on a new episode. We compare three setups. These are delays we chose for a deterministic model on a virtual clock, not measurements of any network or of Netflix:

• Origin only: the viewer talks to an origin with an assumed 250 ms round trip. This is an input we chose to represent a distant origin, not a measured path.
• Pull edge: an edge assumed 10 ms away, starting empty; on a miss it goes to the origin, 240 ms further, and spends 15 ms on lookup and storing.
• Push edge: the same edge, with all 20 chunks placed before play.

Every server spends 5 ms finding a chunk, every body takes 60 ms to transfer on every path (so bandwidth is identical), each hop gets ±10% jitter, the episode is 20 chunks of 2 s, and the player asks for each chunk 300 ms before it is due. Each setup ran 3 times (seeds 11, 22, 33) with the same jitter table shared across setups, and after the first viewer a second viewer watched through the same edge.

Inputs (ms): origin RTT 250, edge RTT 10, edge→origin 240, miss overhead 15,
             server 5, transfer 60, jitter ±10%, lead 300, 20 chunks × 2 s

TTFB, expected without jitter:
  origin only     250 + 5                     = 255
  pull miss       10 + 15 + 240 + 5 + 5       = 275    (edge hop, miss work, origin hop, two servers)
  hit (pull/push) 10 + 5                      = 15

modeled startup = TTFB(chunk 1) + transfer 60   (assumes playback waits for the complete first segment):
  origin only     255 + 60 = 315 expected  → model 318 ms
  pull, viewer 1  275 + 60 = 335 expected  → model 335 ms
  push, viewer 1   15 + 60 =  75 expected  → model 75 ms
  pull, viewer 2   15 + 60 =  75 expected  → model 75 ms

stall check (lab rule): chunk needs TTFB + 60 < 300 lead
  origin  254 + 60 = 314 > 300 → late on most chunks   → 16 stalls, 282 ms rebuffering
  pull v1 273 + 60 = 333 > 300 → late on every chunk   → 19 stalls (of 19 after the first), 628 ms
  hits     15 + 60 =  75 < 300 → 225 ms of slack       → 0 stalls

Averages over 3 runs (model):
  setup / viewer   startup       mean TTFB   stalls   edge hit rate
  origin only        318 ms        254 ms      16          -
  pull, viewer 1     335 ms        273 ms      19          0%
  pull, viewer 2      75 ms         15 ms       0        100%
  push, viewer 1      75 ms         15 ms       0        100%

Read the table in three steps. First, the pull edge did not help the first viewer: 0% hits, a modeled startup 17 ms slower than the origin (335 against 318) and more stalls (19 against 16), because each miss adds the edge hop, the miss work and a second server delay. Second, those misses filled the cache, so viewer 2 got 75 ms and no stalls. Third, the push edge gave 75 ms to the very first viewer. With identical bandwidth, the gap comes from where the chunk was stored plus the pull miss cost.

The tight 300 ms lead makes the difference visible; real players buffer many seconds, so real stalls are rarer and depend on sustained delivery rate. The equal 60 ms transfer is a simplification in both directions: on a real path, transfer time depends on whether the connection is already warm, how the congestion controller ramps its window per round trip, and the bottleneck capacity of each path, so the real gap can be larger or smaller than the model shows.

Code

The first snippet is a trimmed version of the lab's edge cache: fixed delays, no jitter, one chunk, a virtual clock (it just adds numbers). Save it as edge_cache.mjs and run node edge_cache.mjs with Node.js 18 or newer. No packages, no network.

// edge_cache.mjs: origin vs pull vs push on a virtual clock (fixed delays we chose, no network)
const D = { originRtt: 250, edgeRtt: 10, edgeToOrigin: 240, missOverhead: 15, server: 5, transfer: 60 };

class Origin {
  constructor() { this.requests = 0; }
  fetch() { this.requests++; return D.originRtt + D.server; }   // time to first byte, ms
}

class EdgeCache {
  constructor(origin) { this.origin = origin; this.store = new Set(); this.hits = 0; this.misses = 0; }
  prefill(ids) { for (const id of ids) this.store.add(id); }      // push: copy in before anyone plays
  fetch(id) {
    if (this.store.has(id)) { this.hits++; return D.edgeRtt + D.server; }   // HIT: short hop only
    this.misses++;                                                          // MISS: pull from origin
    this.origin.requests++;
    this.store.add(id);                                                     // keep it for the next viewer
    return D.edgeRtt + D.missOverhead + D.edgeToOrigin + D.server + D.server;
  }
}

const firstFrame = (path) => path.fetch(0) + D.transfer;   // chunk #1 only

console.log("origin only     ", firstFrame(new Origin()), "ms");

const pull = new EdgeCache(new Origin());
console.log("pull, viewer 1  ", firstFrame(pull), "ms");
console.log("pull, viewer 2  ", firstFrame(pull), "ms");

const push = new EdgeCache(new Origin());
push.prefill([0, 1, 2]);                                    // done earlier, off-peak, in a real system
console.log("push, viewer 1  ", firstFrame(push), "ms");
console.log("origin requests at play time: pull", pull.origin.requests, "| push", push.origin.requests);

// Coalescing: 100 viewers ask for the same cold chunk in the same instant.
function burst(n, coalesce) {
  const origin = new Origin();
  const inFlight = new Map();                               // chunk id -> fetch already running
  for (let i = 0; i < n; i++) {
    if (coalesce && inFlight.has(7)) continue;              // join the running fetch
    inFlight.set(7, origin.fetch(7));
  }
  return origin.requests;
}
console.log("cold burst of 100: origin requests", burst(100, false), "without lock,", burst(100, true), "with lock");

What to observe:

origin only      315 ms
pull, viewer 1   335 ms
pull, viewer 2   75 ms
push, viewer 1   75 ms
origin requests at play time: pull 1 | push 0
cold burst of 100: origin requests 100 without lock, 1 with lock

Without jitter the origin row is the expected 315 ms; the full lab adds ±10% jitter and averages three runs, giving 318 ms. The pull rows show "miss, then hit" for one chunk, and the push edge sent nothing to the origin at play time. Set missOverhead to 0 and the first pull viewer becomes 320 ms, still slower than 315 because of the extra edge hop and server delay. The coalescing line is the ideal case, with an instant burst and a fill that never fails or times out; the Nginx section below shows when real locks let extra requests through. The full lab, labs/sd-14/run.mjs, plays all 20 chunks and counts stalls.

The same ideas in a real cache: this config makes Nginx a small pull edge with coalescing and stale serving (directives from the proxy module docs):

# edge.conf: Nginx as a pull cache in front of a faraway origin
proxy_cache_path /var/cache/edge levels=1:2 keys_zone=video:100m
                 max_size=50g inactive=7d use_temp_path=off;

server {
    listen 80;

    location /video/ {
        proxy_pass https://origin.example.com;
        proxy_cache video;
        proxy_cache_key "$scheme$proxy_host$request_uri";   # the default: keeps the full query string
        proxy_cache_valid 200 206 7d;               # fallback lifetime; upstream cache-control headers take precedence

        proxy_cache_lock on;                        # coalesce concurrent misses for the same key
        proxy_cache_lock_timeout 5s;                # after 5 s a waiter goes upstream, uncached
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;           # refresh stale copies without making viewers wait

        add_header X-Cache-Status $upstream_cache_status always;   # MISS, HIT, EXPIRED, STALE, UPDATING
    }
}

The cache key is a security boundary. The key above is the documented default, $scheme$proxy_host$request_uri, which includes the query string. A common "optimization" is $scheme$host$uri, which drops every query parameter. That is unsafe: two different representations (say ?rendition=1080p and ?rendition=480p) collide on one key, and with signed URLs, a request carrying a valid signature fills an entry that a later request with an expired or missing signature is then served from cache, because the origin that checked the signature is never asked. Keeping the signature in the key does not fix this either: it only fragments the cache, and expiry is still not enforced once a copy is stored. Protected content needs authorization before the cache lookup, at the edge itself, for example with ngx_http_secure_link_module checking the hash and expiry, or an auth_request subrequest. Only strip a query parameter from the key after you have proven it changes neither the content nor access control (analytics tags such as utm_* are the usual candidates).

Request the same segment twice with curl -sI http://localhost/video/ep1/seg-001.ts and watch X-Cache-Status go from MISS to HIT; compare curl -w '%{time_starttransfer}\n' -o /dev/null -s <url> between the two to see TTFB drop. To "push" with a pull cache, warm it: request every segment of a new title from a script before it is announced. That is the simplest form of cache warming.

How Nginx stores the cache, and sizing it

The directives above hide a storage engine with a specific lifecycle. The mechanics below follow the proxy_cache_path documentation and Nginx's cache implementation; the sizing numbers are our worked assumptions.

Filenames come from an MD5 of the key. Nginx hashes the cache key with MD5 and uses the 32-character hex digest as the filename. levels=1:2 builds a two-level directory tree from the end of the hash: the last character, then the two before it. The docs' example: key hash b7f54b2df7773722d382f4809d65029c is stored at /data/nginx/cache/c/29/b7f54b2df7773722d382f4809d65029c. That spreads files over 16 × 256 = 4,096 directories, so no single directory holds millions of entries.

Metadata lives in shared memory. keys_zone=video:100m creates a shared memory zone, visible to every worker process, holding all active keys and per-entry metadata (validity, last access). The docs give one megabyte as about 8 thousand keys, so 100 MB indexes about 800,000 entries. Inside the zone, entries sit in a red-black tree keyed by the MD5 of the cache key, so a lookup is O(log n) in shared memory, and a queue orders the same entries by last access, which is what eviction walks from the cold end. A hit still touches disk before the body: Nginx opens the file and reads its header (format version, validity times, a CRC32 of the key and the key itself) to validate the entry, and treats a mismatch as a miss. The zone is a second capacity limit beside max_size, so size it from your expected object count, not from disk size.

Fills go through a temporary file, then a rename. An upstream response is first written to a temporary file and then renamed into the cache, so the entry appears on disk only once the body is complete. With use_temp_path=off the temporary file is written inside the cache directory, which keeps the rename on one filesystem (a cheap metadata operation instead of a copy).

Two helper processes manage the lifecycle. The cache loader runs once after startup and walks the cache directory to rebuild the shared-memory metadata from files on disk, in batches (loader_files, default 100 files per iteration, at most loader_threshold 200 ms per iteration, loader_sleep 50 ms between). Objects already on disk are not lost while it runs: during this cold-loading phase, a lookup that finds no entry in shared memory still checks for the file on disk and, if a valid one exists, indexes and serves it on demand. The cost of a restart is therefore extra filesystem work (open and header reads for keys the loader has not reached yet, competing with the loader's own directory walk) and some startup latency, not an automatic drop in hit ratio. The cache manager runs periodically: when total size exceeds max_size (or free space drops below min_free), it removes least recently used data.

Freshness and inactive are different clocks. Freshness (Cache-Control from the origin, else proxy_cache_valid) decides whether an entry can be served without asking upstream. inactive decides whether it stays on disk at all: an entry not accessed within that time is removed regardless of its freshness (default 10 minutes). So with proxy_cache_valid 7d and the default inactive, a segment watched once and then ignored for 11 minutes is gone, despite being fresh for a week. Conversely, an expired but often-accessed entry stays on disk and can be revalidated with a cheap 304. The config above sets inactive=7d so that long-tail segments survive between viewers.

Worked sizing. Assume a 6 Mbit/s rendition and 2-second segments:

segment size  = 6 Mbit/s × 2 s ÷ 8 bits per byte   = 1.5 MB
10,000 segments × 1.5 MB                           = 15 GB   (one rendition, 20,000 s ≈ 5.6 h of video)
max_size=50g is binary: 50 GiB = 50 × 1024³        = 53,687,091,200 bytes
53,687,091,200 ÷ 1,500,000 bytes                   ≈ 35,791 segments
53,687,091,200 ÷ 15,000,000,000 bytes              ≈ 3.58 catalogs
keys needed ≈ 35,791  vs  zone capacity ≈ 800,000  → disk, not the key zone, is the binding limit

Nginx's size parser uses binary multipliers, so g means GiB while media sizes above are decimal. So this 50 GiB edge holds about 3.58 such single-rendition catalogs, before per-file cache headers and filesystem block overhead; plan for less and keep operational headroom below max_size and the disk size. Adaptive bitrate multiplies the footprint: every rendition is a separate set of keys, so three renditions of the same 5.6 hours already need 45 GB if all are watched. Fill bandwidth is the other side. Pushing 15 GB into one edge in a 6-hour off-peak window needs 15 × 8 = 120 Gbit ÷ 21,600 s ≈ 5.6 Mbit/s sustained per edge; across 1,000 edges filled from one shield, that is about 5.6 Gbit/s leaving the shield for six hours.

What to monitor: disk usage against max_size (sitting at the ceiling means the manager is evicting continuously), the rate of MISS for objects that were previously HIT (refills after eviction, the signature of a cache smaller than its working set), byte hit ratio rather than request hit ratio, and upstream request counts per key during launches. All of these come from logging $upstream_cache_status, $request_uri and $body_bytes_sent in the access log.

Failure modes and edge cases

• Cold start after a release. On a pure pull CDN a new title has zero hits everywhere at launch, and the first viewers at each edge all miss together. Detect it as a spike in origin requests right after launch; fix it with pre-positioning, warming, a shield and coalescing.
• Cache stampede on expiry. A popular object expires everywhere at once and the next burst all misses. Use coalescing, stale-while-revalidate, immutable versioned URLs, or random jitter on TTLs.
• Cache key mistakes. A per-user query parameter in the key makes every request unique and the hit ratio collapses; leaving out something that changes the response serves the wrong version, and dropping a signed-URL query string can serve protected content without the check. Review keys and Vary headers explicitly, and authorize before the cache lookup.
• Caching personal data. A response missing private can be stored in a shared cache and served to another user. Mark authenticated responses private or no-store and test it.
• Eviction churn. If the edge is smaller than the working set, popular objects push each other out under load. Watch eviction rate and byte hit ratio.
• Fill traffic competing with viewers. Pushing uses the same links viewers use, which is why an off-peak fill window, as in the Open Connect overview, matters.
• Origin outage. A pull CDN with short TTLs fails fast when the origin dies, because misses cannot be filled. stale-if-error and long TTLs on immutable objects keep the service up; the same availability thinking as in the CAP article on what to return when a link breaks.

Trade-offs

Common mistakes

In an interview

1. Clarify: video or small files? On demand or live? How many regions, how big is the catalog, how skewed is popularity?
2. Assumptions: state round trips to the origin and to an edge, segment size and duration, and say they are assumptions.
3. Structure: player and manifest, routing to an edge, edge cache, optional shield, origin; push fill path beside the pull path.
4. Deep dive: miss cost, hit ratio arithmetic, cache keys and Cache-Control, coalescing, what is pushed and when.
5. Trade-offs: storage versus first-view latency, freshness versus hit ratio, shield hop versus origin protection.
6. Wrap up: most requests are hits from nearby; the design work is making the first request for important content a hit too, without wasting storage.

Practice

Cheat sheet

Further reading

• Netflix Open Connect Overview (PDF): the public description of Open Connect, including proactive content placement and the off-peak fill window.
• Netflix Open Connect: the program's site, with material for internet providers that host appliances.
• RFC 9111, HTTP Caching: the rules every shared cache follows: freshness, age calculation, validation and Cache-Control directives.
• MDN, Cache-Control: a readable reference for each directive, including immutable, stale-while-revalidate and stale-if-error.
• Cloudflare Learning, What is a CDN?: an overview of edges, routing and why distance matters, with diagrams.
• Nginx ngx_http_proxy_module: the exact meaning of proxy_cache, proxy_cache_lock, proxy_cache_use_stale and related directives used in the config above.

This article describes general CDN and HTTP caching behaviour and our own small simulation. Apart from the one cited statement from Netflix's public Open Connect overview, it does not describe any company's private systems, and the diagrams are conceptual. The simulated numbers come from a deterministic model with delays we chose; they show the shape of the effect, not measurements of Netflix, of any CDN or of any network in Nepal.