# Eventual consistency: stale likes and safe withdrawals
URL: https://systemdesignsimple.com/like-count-wrong-balance-never/
Published: 2026-11-05
Category: System Design
Series: System Design, simply, Part 8: Consistency models: eventual, quorum, linearizable
Publisher: System Design Simple (https://systemdesignsimple.com)
## Short answer
A like count can be a few behind because reads go to copies that lag, and that is cheap. For a balance, an atomic conditional debit prevents overdrafts under the stated assumptions: one serial place checks and applies each withdrawal in one step, and fresh reads are not what protects it.
## Key facts
- **Eventual reads in our simulation:** 59.2% of reads were stale (median 2 likes behind); reads and writes both had a median of 2 ms; followers matched the leader 46.9 ms after the last write
- **Weak reads in our simulation:** 66.3% stale; lost replication messages were never resent, so one follower stayed 13 likes short for good
- **Quorum W=2, R=2 in our simulation:** 0% of reads missed a like acknowledged before the read started; read median 2.2 ms; write median 111.3 ms, p99 188.5 ms
- **Quorum is not linearizable:** 26.1% of quorum reads returned fewer likes than a read that had already finished: W + R > N only covers writes acknowledged before the read started
- **Balance overdrafts:** With a client-side check: weak 320, eventual 302, strong 143. With an atomic check at the leader: 0 in every mode
- **What prevents overdrafts:** One serial place checks the rule and applies the change atomically (a conditional write, a compare-and-set, SELECT ... FOR UPDATE with the check and debit inside the same transaction, or a serializable transaction with retries), with acknowledged withdrawals preserved through failover
- **Setup:** Our simulation, labs/sd-8: seed 42, virtual clock, 3 replicas, leader never fails, Node v22.13.1, Apple M2 Pro. Not a benchmark
Eventual consistency allows reads from replicas that have not received the newest writes. When one value lives on several replicas, a read can land on such a copy and return an old value. A like count can live with that; a balance cannot. What stops an overdraft is one place that checks and applies each withdrawal atomically, and a fresher read alone does not. A common interview question puts it as: why can a like count be wrong while a withdrawal never overdraws?
## Prerequisites {#prerequisites}
**Leader and followers.** In leader-based replication one server, the leader (primary), accepts every write and sends each change to the followers (replicas, the other servers holding a copy), which also serve reads. The delay before a change arrives is replication lag, covered from the read-scaling side in [one database writes, ten read: primary and read replicas](/one-database-writes-ten-read/).
**Check then act.** "If the balance is at least 100, take 100" is two steps; if two requests both check before either acts, both pass. [Two fans clicking the same seat](/two-fans-one-seat/) showed the fix: check and change in one atomic step in one place.
**What a partition forces.** [When the cable to Asia breaks](/cable-to-asia-breaks/), a system must choose between answering from a copy that may be behind and refusing. This article stays on a healthy network: what does each read mode promise, and what does it cost?
## Definitions {#definitions}
> Definition: **Acknowledged (acked) write**: a write the system has confirmed to the client as done, either as soon as the leader has it or only after a quorum of replicas has stored it.
> Definition: **Stale read**: in this article, a read that returns fewer likes than the writes already acknowledged to clients when the read started. This is the lab's exact definition.
> Definition: **Weak consistency**: after a write, the system does not guarantee that later reads will return it, ever. In our lab, "weak" means a lost replication message is never resent, so a follower can stay wrong for good.
> Definition: **Eventual consistency**: if no new writes arrive, all replicas eventually return the last written value. It says nothing about how stale a read is before that point.
> Definition: **Quorum (N, W, R)**: N copies of a value; a write is acked after W of them store it; a read asks R of them and keeps the newest version. When W + R > N, every read set shares at least one replica with every write set that acked.
> Definition: **Linearizability**: every operation appears to take effect at one instant between its start and its finish, as if there were a single copy, in an order that respects real time. Once a read has returned a value, any read that starts later returns that value or a newer one.
> Definition: **Went back**: in our lab, a read that returns fewer likes than some read that had already finished before it started. A linearizable system never does this.
> Definition: **Atomic conditional write**: a write that checks a condition and applies the change as one indivisible step, so no other write can slip between the check and the change. Also called compare-and-set when the condition is "the value is still what I read".
## How eventual consistency returns an old like count {#how-it-works}
Every like goes to the leader first. The leader adds one to its count and sends the change to both followers. Each follower applies it after a delay. In our lab that delay is random, anywhere from 0 to 200 ms, and 2% of the forwarding messages are lost on the way.
A read goes to one random replica. On the leader it sees every like; on a follower, only the likes that have arrived so far. With a like every 50 ms on average and each copy taking up to 200 ms, a follower almost always has a few likes in flight.
That gives a ceiling before running anything: about a third of reads land on the leader and are never stale. With uniformly random routing, the expected stale share is at most two thirds (about 66.7%); a finite sample can exceed that.
Our lab (labs/sd-8, a one-machine simulation, not a database benchmark) ran 20 simulated seconds at 20 likes and 200 reads per second, the same workload in every mode (413 likes and 3,940 reads per mode in the printed run):
- **Eventual** (lost messages resent after 100 ms until they arrive): 59.2% of reads were stale, a median of 2 likes behind when stale. Reads took a median of 2 ms, and so did writes, because the ack comes straight from the leader.
- **Weak** (lost messages never resent): 66.3% of reads were stale, very close to the two-thirds ceiling. One follower ended 13 likes short and never caught up, because nothing ever resent the missing changes.
Little's law checks the "2 likes behind": items in a system = arrival rate × average time inside. At 20 likes per second and an average lag of about 100 ms (the middle of 0 to 200 ms), about 20 × 0.1 = 2 likes are in flight to a follower. It is an approximation, but it matches the measured median.
The modes also differed in convergence. Eventual followers matched the leader 46.9 ms after the last write (replication lag p99 198.2 ms, at the edge of the injected 0 to 200 ms). Weak followers made their last update 124.4 ms after the last write and then stopped while still different, so they never converged.
Werner Vogels' 2008 article describes this family: if no new updates are made, eventually all accesses return the last value, and the "inconsistency window" depends on load, network delay and replica count. He also lists cheaper, stronger variants such as read-your-writes (you always see your own like) and monotonic reads (your reads never go back), which matter more for a like button than full strong consistency.
## What does W + R > N actually guarantee? {#quorum}
The quorum idea comes from designs like Dynamo (2007 paper): keep N copies, ack a write after W store it, read R and keep the newest. The paper notes that R + W > N "yields a quorum-like system", with N = 3, R = 2, W = 2 a common setting. Apache Cassandra exposes the same idea as the `QUORUM` consistency level.
Our lab's strong mode is exactly that: a like is acked after the leader and one follower store it (W = 2), a read asks two random replicas and keeps the higher version (R = 2), and followers apply in order.
The overlap follows from counting. A write sits on at least 2 of 3 replicas; a read asks 2 of 3; since 2 + 2 = 4 > 3, the two groups share a replica, and "keep the newest" picks the write.
The precise promise is that a quorum read sees every write that was acknowledged before the read started. In the lab, quorum reads missed such a write 0% of the time, and they were cheap: a median of 2.2 ms against 2 ms (p99 2.9 ms), because replicas answer at once and the client waits only for the slower of two round trips.
The cost moved to writes. A quorum write waited for a follower: median 111.3 ms instead of 2 ms, p99 188.5 ms, about 111.3 / 2 = 56 times slower at the median.
If followers applied in any order, the ack would wait for the faster of two lags, each uniform from 0 to 200 ms, whose median is 200 × (1 - √0.5) ≈ 58.6 ms. But strong followers apply in order, so a quick like waits behind an earlier slow one, and a lost message adds a 100 ms resend. In our run that nearly doubled the wait.
The overlap argument does not cover a write still in progress: one stored on the leader but not yet acknowledged may or may not be seen, and that gap lets a count go backwards.
## Why zero stale reads is still not strong consistency {#linearizable}
Here is the case from the video, concept numbers only. The leader has just applied like number 12 but has not acked it, because no follower has stored it yet. Both followers still have 11.
```text
replica leader F1 F2
holds 12 11 11 (like 12 not yet acked)
read 1 asks leader + F1 -> newest is 12 finishes
read 2 starts after read 1 has finished
asks F1 + F2 -> newest is 11
```
Neither read is stale by our definition: like 12 was never acknowledged, so read 2 did not have to see it. Yet a user watching saw 12 and then 11, so the count went backwards.
Linearizability forbids exactly that. Herlihy and Wing's 1990 definition says each operation must appear to take effect at one instant between its start and finish, in an order that matches real time. Read 1 finished with 12, so like 12 had taken effect; read 2 started later, so it must see 12 or more.
In the lab, 26.1% of quorum reads went back. The quorum run never missed an acknowledged like and was still not linearizable: "zero stale still wasn't strong".
Real systems close the gap in a few standard ways:
- Read from a leader that has confirmed it still leads, with a majority or a time-bound lease, as consensus systems such as Raft do.
- Write back before returning. The classic ABD register algorithm makes a reader write the newest value it found to a quorum before answering, so later reads must see it. Read repair done before answering has the same effect, at the cost of read latency.
- Order every write through one log, and let a read wait until its replica has applied up to a known position.
Each puts waiting somewhere. Jepsen's linearizability page also notes it cannot be totally available: during a partition some requests must wait or fail, the CAP trade-off from [Part 7](/cable-to-asia-breaks/). For likes none of this is worth it.
## What actually keeps a balance correct? {#balance}
A common answer is that overdrafts are prevented "because banks use strong reads". The lab shows that this answer is incomplete.
Run 2 uses 300 accounts, each starting at 300 and hit by 6 withdrawals of 100 a few tens of milliseconds apart (mean gap 60 ms). Rule: withdraw only if the balance is at least 100. Each account can honestly pay 3, so the correct total is 300 × 3 = 900 approvals. An overdraft is an approved withdrawal that takes the real balance below zero. Two experiments, each in all three read modes:
**Client check.** The client reads the balance (one replica, or two in strong mode), checks the rule, then sends a plain "minus 100". Overdrafts: weak 320, eventual 302, strong 143. Quorum reads cut overdrafts by more than half but did not stop them:
1. A quorum read only covers acknowledged writes. A withdrawal the leader has applied but not acked is invisible to a read that asks the two followers, the same gap as like 12.
2. Even a perfectly fresh read is a snapshot. Two withdrawals can both read 100, both pass, and both send "minus 100". That is the check-then-act race from the seat example; no read mode fixes it, because the problem is the gap between reading and writing.
**Leader check.** Same read and client check, then the leader checks the rule again in one serial place and applies the withdrawal atomically. Overdrafts: 0 in weak, eventual and strong; every mode approved exactly 900. In strong mode the leader stopped 159 withdrawals the client's check had passed (360 weak and 329 eventual in the printed run, because those reads were staler).
The answer to the title, then: an atomic conditional debit prevents overdrafts under the stated assumptions, with one serial place checking the rule and applying the change in one step. The atomic step is a conditional write, a compare-and-set, a `SELECT ... FOR UPDATE` row lock with the check and the debit inside the same transaction, or a serializable transaction with retries. A transaction that does an ordinary read and then an unconditional decrement can still race at Read Committed. Quorum reads are not the protection.
The storage under the check still has to be strong. In our lab the leader never fails, which is why its check worked even in weak mode. In a real system, if the leader dies and a follower missing the last acknowledged withdrawals takes over, its check runs on an old balance. Preserve acknowledged withdrawals through durable replication, promote only a replica containing them, and fence the old leader. A correctly configured consensus protocol coordinates these guarantees. The atomic check stops races; failover-safe storage keeps the checker correct across failures.
In PostgreSQL, `UPDATE ... WHERE balance >= 100` locks the row it changes. The transaction isolation docs explain that at the default Read Committed level a second updater of that row waits for the first to commit, then re-evaluates its `WHERE` clause against the new row version; if the balance is now below 100, zero rows change. DynamoDB offers the same idea as condition expressions, Cassandra as lightweight transactions (`IF` conditions).
## Worked example {#worked-example}
The video's scenario, concept numbers only: your phone shows 10 likes, your friend's shows 12, and both are telling the truth about the replica they asked. Then the two-read example: the leader holds an unacked 12, followers hold 11, read 1 gets 12, read 2 starts after it and gets 11.
The measured version, from our lab (labs/sd-8, seed 42, virtual clock, 3 replicas, leader never fails, Node v22.13.1, Apple M2 Pro). Assumptions printed by the lab: follower lag uniform 0 to 200 ms, 2% of forwarding messages lost, resend after 100 ms (eventual and strong), client one-way delay 0.5 to 1.5 ms.
```text
Run 1: like counter, 20 likes/s and 200 reads/s for 20 simulated seconds
ceiling for stale reads with 1-replica reads: 2 of 3 replicas lag -> about 66.7%
in flight per follower (Little's law): 20 likes/s x 0.1 s mean lag = about 2 likes
weak eventual strong (W=2, R=2)
stale reads 66.3% 59.2% 0% (vs writes acked before read start)
median behind (stale) 7 2 0
went back 66.1% 54.8% 26.1%
read p50 2 ms 2 ms 2.2 ms (p99 2.9 ms)
write p50 2 ms 2 ms 111.3 ms (p99 188.5 ms)
likes lost on a follower 13 0 0
converged after last never 46.9 ms 158.5 ms
write (last follower update 124.4 ms, followers still differ)
write cost: 111.3 / 2 = about 56 times slower at the median
```
```text
Run 2: balance, 300 accounts x 6 withdrawals of 100, start balance 300
correct approvals: 300 accounts x (300 / 100) = 900
client check, then plain "minus 100"
weak approved 1220 overdrafts 1220 - 900 = 320 money missing 320 x 100 = 32000
eventual approved 1202 overdrafts 1202 - 900 = 302 money missing 30200
strong approved 1043 overdrafts 1043 - 900 = 143 money missing 14300
atomic check at the leader
weak approved 900 overdrafts 0
eventual approved 900 overdrafts 0
strong approved 900 overdrafts 0 (159 stopped at the leader)
```
Stronger reads cost latency and cut stale reads, but only the atomic check took overdrafts to zero, in every mode. These one-machine simulation numbers depend on the chosen lag, loss and request rates; they measure no real database.
## Code {#code}
This Python script builds a like counter with one leader and two lagging followers, compares one-replica reads with quorum reads (W = 2, R = 2), and ends with the two-read example. Standard library only: `python3 likes_lab.py`. It is smaller than the lab (no lost messages, no network delay, instant reads, a like every 20 ms), so its numbers differ; the pattern is the same.
```python
# likes_lab.py: one leader, two lagging followers, stale reads vs quorum reads
import random
import statistics
random.seed(8)
LIKES, GAP_MS, LAG_MS = 200, 20, (0, 200) # one like every 20 ms, follower lag 0 to 200 ms
READS = 2000
# 1. Writes. The leader (replica 0) applies like i at time t_i. Each follower applies
# it later, in order (never before the like before it).
t = [i * GAP_MS for i in range(LIKES)]
applied = [t[:], [], []] # applied[r][i] = when replica r stores like i
for f in (1, 2):
last = 0
for i in range(LIKES):
last = max(last, t[i] + random.uniform(*LAG_MS))
applied[f].append(last)
# Ack times. Eventual: as soon as the leader has it. Quorum W=2: once a follower also has it.
ack_eventual = t[:]
ack_quorum = [min(applied[1][i], applied[2][i]) for i in range(LIKES)]
def value(r, now): # likes replica r holds at time `now`
return sum(1 for x in applied[r] if x <= now)
def acked(acks, now): # likes already confirmed to clients
return sum(1 for x in acks if x <= now)
# 2. Reads at random moments (instant reads, so "finished before" means "earlier").
times = sorted(random.uniform(0, t[-1]) for _ in range(READS))
def run(mode):
stale, behind, went_back, seen_max = 0, [], 0, -1
for now in times:
if mode == "eventual":
v = value(random.randrange(3), now) # ask one replica
must = acked(ack_eventual, now)
else:
a, b = random.sample(range(3), 2) # ask two, keep the newer
v = max(value(a, now), value(b, now))
must = acked(ack_quorum, now)
if v < must:
stale += 1
behind.append(must - v)
if v < seen_max: # less than a read that already finished
went_back += 1
seen_max = max(seen_max, v)
med = statistics.median(behind) if behind else 0
print(f"{mode:9} stale {100 * stale / READS:5.1f}% median behind {med:g} "
f"went back {100 * went_back / READS:5.1f}%")
run("eventual")
run("quorum")
wait = [ack_quorum[i] - t[i] for i in range(LIKES)]
print(f"quorum write ack waits a median {statistics.median(wait):.1f} ms for a follower")
# 3. The two-read example: leader has 12 (not yet confirmed), both followers have 11.
state = {"leader": 12, "F1": 11, "F2": 11}
print("read 1 asks leader + F1 ->", max(state["leader"], state["F1"]))
print("read 2 asks F1 + F2 ->", max(state["F1"], state["F2"]), "(started after read 1 ended)")
```
What it prints (Python 3, fixed seed, so every run is the same):
```text
eventual stale 65.7% median behind 7 went back 65.7%
quorum stale 0.0% median behind 0 went back 32.9%
quorum write ack waits a median 119.1 ms for a follower
read 1 asks leader + F1 -> 12
read 2 asks F1 + F2 -> 11 (started after read 1 ended)
```
What to observe:
- Eventual stale is near the two-thirds ceiling. The median gap is 7 instead of the lab's 2 because the toy writes 50 likes per second: about 50 × 0.1 = 5 in flight, and in-order apply adds more. Stale follower reads almost always land below an earlier leader read, so "went back" and "stale" nearly coincide.
- Quorum stale is exactly 0%: with in-order apply, the follower that acked a like holds every earlier like, and any two replicas include the leader or that follower.
- Quorum still went back 32.9%, and the last two lines show why.
- Acks wait a median 119.1 ms. Set `LAG_MS` to `(0, 20)` and both the wait and the stale share fall; set `GAP_MS` to `200` and the stale share falls, with fewer likes in flight.
The balance fix needs no simulation. This is the atomic conditional withdrawal, for PostgreSQL:
```sql
-- withdraw.sql: check and apply in one atomic step, in one serial place
UPDATE accounts
SET balance = balance - 100
WHERE id = $1 AND balance >= 100;
-- 1 row updated = approved
-- 0 rows updated = not enough money, decline
-- a concurrent withdrawal on the same row waits, then re-checks balance >= 100
```
If application code must read, decide, then write, lock the row first with `SELECT balance FROM accounts WHERE id = $1 FOR UPDATE`, and keep the check and the debit inside that same transaction; the explicit locking docs describe how that lock blocks other updates until the transaction ends. The full lab (`node labs/sd-8/run.mjs`) runs both balance experiments in all three modes.
## Failure modes and edge cases {#failure-modes}
**Leader failover with asynchronous replication.** If the leader alone acks writes and then dies, the new leader may lack the last acknowledged writes: a few lost likes, or an atomic balance check running on an old balance. Preserve acknowledged writes through durable replication, promote only a replica containing them, and fence the old leader. A correctly configured consensus protocol coordinates these guarantees.
**Lost messages never resent.** The lab's weak mode: one follower stayed 13 likes short forever. Real systems use retries, anti-entropy (background comparison, which the Dynamo paper does with Merkle trees) and read repair. Monitor for followers that still differ long after writes stop, not only for lag.
**Sloppy quorums.** Some Dynamo-style stores accept a write on fallback nodes when the usual replicas are unreachable and hand it back later (hinted handoff). The W acking nodes may not overlap the R nodes a later read asks, so the overlap argument breaks.
**Last-writer-wins and clock skew.** With timestamp conflict rules and drifting clocks, a newer write can lose. For counters, merge per-replica increments instead of writing the whole count.
**Retried withdrawals.** A client times out and retries, but the first try succeeded; the atomic check approves the second too while money is left. Store a unique idempotency key with each debit so a retry returns the first result.
**Your own like disappearing.** You tap like, the next read hits a lagging follower, and your heart turns grey again, which users notice far more than a count 2 behind. Give that user read-your-writes (leader reads briefly after a write).
**Lag spikes.** A slow disk, a long transaction or a burst like a [million likes at once](/million-likes-at-once/) can push lag from milliseconds to seconds. Alert on lag in seconds and on each follower's log position behind the leader.
## Trade-offs {#trade-offs}
| Option | Good for | Costs you | Use when |
| --- | --- | --- | --- |
| Weak (no resend) | Throwaway data: a typing indicator, a live viewer estimate | Copies can stay wrong forever | Losing an update is truly harmless |
| Eventual, one-replica reads | Like counts, view counts, feeds, profile data | Stale reads inside the lag window; counts can go back | Fast reads matter and a few-behind value is fine |
| Eventual plus read-your-writes | Anything a user just changed themselves | Routing logic or session tracking | Users must see their own action at once |
| Quorum W + R > N | Never missing an acked write without a single leader in the read path | Slower writes (111.3 ms vs 2 ms median in our lab); still not linearizable | Acked writes must be visible and occasional going back is acceptable |
| Linearizable reads (leader with lease, read index, write-back) | Locks, leader election, configuration, uniqueness | Extra round trips; unavailable during a partition | Every reader must agree on one order in real time |
| Atomic conditional write on strongly consistent storage | Balances, stock, seats, any rule like "never below zero" | All writes for a key go through one serial place; that place can become a hot spot | An invariant must hold on every write |
## Common mistakes {#common-mistakes}
- **Mistake:** "We use quorum reads, so reads are strongly consistent." **Instead:** say what W + R > N covers: writes acknowledged before the read started. Reads can still go back, as 26.1% did in our simulation. Linearizable reads need extra work such as leader reads or write-back.
- **Mistake:** protecting a balance by reading it fresh, checking in application code, then sending a plain decrement. **Instead:** make the check part of the write, as in `UPDATE ... WHERE balance >= 100`, or lock the row first. In our lab, that change alone took overdrafts from 143 to 0 in strong mode.
- **Mistake:** paying for strong consistency on every counter. **Instead:** classify data by the cost of a wrong value: eventual likes, an atomic rule on strong storage for balances.
- **Mistake:** treating "eventually" as a time bound. **Instead:** measure lag and convergence, and retry lost messages, or "eventually" becomes "never" as in the lab's weak mode.
- **Mistake:** assuming the atomic check alone is enough when the leader can fail. **Instead:** preserve acknowledged withdrawals through durable replication, promote only a replica containing them, and fence the old leader, so the next leader's check still runs on the true balance. A correctly configured consensus protocol coordinates these guarantees.
## In an interview {#in-an-interview}
1. **Clarify:** which data is this? A like count and a balance have different costs when wrong. Ask whether users must see their own writes at once and which rules must never break.
2. **Assumptions:** a leader with followers in one region, replication lag of tens to hundreds of milliseconds, three copies of each value.
3. **Structure:** likes go to the leader and are read from any replica; withdrawals go to the primary database as one conditional write.
4. **Deep dive:** explain stale reads and the lag window, then W + R > N and its exact guarantee, then why a quorum read can still go back, then the check-then-act race on a balance.
5. **Trade-offs:** quorum writes are much slower (111.3 ms vs 2 ms median in a simulation), linearizable reads cost extra round trips and availability, and a single serial place per account can become hot.
6. **Wrap up:** eventual plus read-your-writes for likes; an atomic conditional debit on strongly consistent, failover-safe storage for the balance; idempotency keys for retries.
> **A 60-second answer**A like count is read from replicas that copy the leader with a delay, so a read can return an old number. That is eventual consistency, and it is the right trade: in a small simulation, 59.2% of reads were a couple of likes behind, but reads and writes took about 2 ms. Quorum reads with W plus R greater than N never miss a write acknowledged before the read started, but writes slowed to 111.3 ms at the median and 26.1% of reads still went backwards, so quorum is not linearizable. Overdrafts are prevented for a different reason: each withdrawal is one atomic conditional write in one serial place, such as UPDATE ... WHERE balance >= 100, on storage that keeps acknowledged writes through a failover. In the same simulation a client-side check still allowed 143 overdrafts with quorum reads; the atomic check allowed zero in every mode.
## Practice {#practice}
**Q (Beginner): Why can a read return an old like count even when nothing has failed?**
Followers apply the leader's changes after a delay. A read that lands on a follower during that delay sees only the likes that have arrived so far.
**Q (Beginner): What is the difference between weak and eventual consistency in the lab?**
Eventual resends lost replication messages until they arrive, so followers match the leader once writes stop (46.9 ms after the last write). Weak never resends, so a follower stayed 13 likes short forever.
**Q (Intermediate): With reads spread evenly over one leader and two followers, what is the most that one-replica reads can be stale, and why?**
About two thirds (66.7%). Reads that hit the leader are never stale, and a third of reads hit it. Weak mode, where followers fall permanently behind, reached 66.3%.
**Q (Intermediate): N = 3, W = 2, R = 2. Prove a read sees every write acked before it started.**
The write is stored on at least 2 of 3 replicas and the read asks 2 of 3. Since 2 + 2 > 3, the two sets share a replica, and keeping the newest version returns the write.
**Q (Intermediate): Use Little's law to estimate how many likes are in flight to one follower at 20 likes per second and an average lag of 100 ms.**
In flight = arrival rate × time in transit = 20 × 0.1 = 2 likes, which matches the lab's median of 2 likes behind for stale eventual reads.
**Q (Advanced): Quorum reads never missed an acked like. Why did 26.1% still go back?**
A read can see a write the leader has applied but not acked yet. A later read that asks the two followers misses it and returns less. That is allowed by W + R > N but forbidden by linearizability.
**Q (Advanced): Quorum reads cut overdrafts from 302 to 143. Why not to zero?**
Two reasons: the read only covers acked withdrawals, so one in progress can be invisible, and two clients can read the same balance before either decrement lands. The check and the change are separate steps. An atomic check at the leader took overdrafts to 0 in every mode.
**Q (Advanced): The atomic check worked even in weak mode in the lab. Why would it fail in production with weak replication?**
The lab's leader never fails. In production, if the leader dies and a follower missing acked withdrawals takes over, the new leader checks the rule against an old balance and can approve money that is gone.
## Cheat sheet {#cheat-sheet}
| Term or rule | Meaning |
| --- | --- |
| Stale read | Returns less than the writes acked before the read started |
| Weak consistency | No promise later reads see a write, ever |
| Eventual consistency | With no new writes, all copies end up equal; no time bound |
| Read-your-writes | A user always sees their own writes |
| Monotonic reads | A user's later reads never show older data than earlier ones |
| Quorum N, W, R | Ack after W of N store it; read R copies, newest wins |
| W + R > N | Read sees every write acked before it started; nothing more |
| Linearizability | Acts like one copy; a read never returns less than a read that already finished |
| Went back | Returned less than a read that finished before it started |
| Little's law | Items in flight = arrival rate × average time inside |
| Check then act race | Two requests both pass a check before either applies its change |
| Atomic conditional write | Check and change in one step: UPDATE ... WHERE balance >= amount, compare-and-set |
| SELECT ... FOR UPDATE | Locks the row so others wait until your transaction ends |
| Idempotency key | Unique id per operation so a retry is not applied twice |
For availability arithmetic, see 99.9% sounds perfect, but it allows 8.76 hours down in a 365-day year, and how each extra nine cuts that by ten.
## Further reading {#further-reading}
- **Jepsen, Linearizability:** a precise statement of the model and why it cannot stay totally available.
- **Vogels, Eventually Consistent (2008):** the inconsistency window, N, W and R, and variants such as read-your-writes.
- **Herlihy and Wing (1990):** the original definition, with the real-time rule used in the two-read example.
- **The Dynamo paper (2007):** N, R and W quorums, sloppy quorums, hinted handoff and Merkle-tree anti-entropy.
- **PostgreSQL, Transaction Isolation:** how Read Committed re-checks an UPDATE's WHERE clause after a concurrent update.
- **PostgreSQL, Explicit Locking:** row locks such as SELECT ... FOR UPDATE and what each blocks.
- **Designing Data-Intensive Applications:** replication lag, quorums, why quorums are not linearizable, and consensus.
This article describes general patterns and our own small simulation. It does not describe any bank's or company's private systems. The simulated numbers show the shape of the trade-off, not production measurements.
## Frequently asked questions
**Why can a like count show an old number?**
Because the count is copied from a leader to followers after a delay, and a read can land on a follower that has not received the newest likes yet. That is eventual consistency: the copies agree once writes stop. For likes it is a good trade, since reads stay fast and being a few behind for a fraction of a second harms no one.
**Does W + R > N make reads strongly consistent?**
No. It makes every read set overlap every set that acknowledged a write, so a read sees every write acknowledged before the read started. It does not stop a read from seeing a write that is still in progress while a later read misses it. In our simulation 26.1% of quorum reads went back like that, so quorum reads were not linearizable.
**What does linearizable mean in plain words?**
The system acts like one copy of the data, and every operation takes effect at one instant between its start and its finish. So once any read has returned a value, every read that starts later sees that value or a newer one. It is the strongest single-object consistency model.
**So what stops a withdrawal from overdrawing a balance?**
The withdrawal is not a read, then a separate write. One serial place, usually the primary database, checks the balance and applies the debit in one atomic step, such as UPDATE ... WHERE balance >= 100. Two withdrawals cannot both pass on the same balance. In our simulation, under its stated assumptions, that check stopped every overdraft in every read mode.
**If an atomic check fixes overdrafts, why use strongly consistent storage at all?**
Because the atomic check is only as good as the data it runs on. In our lab the leader never fails, so its check always saw every accepted withdrawal. In real systems leaders fail, and if the new leader is missing acknowledged writes, its check runs on old data. Preserve acknowledged withdrawals through durable replication, promote only a replica containing them, and fence the old leader. A correctly configured consensus protocol coordinates these guarantees.
**Should likes use quorum reads?**
Usually not. In our simulation quorum writes took a median of 111.3 ms instead of 2 ms, and the count still went backwards for 26.1% of reads. A typical like-counter design can accept eventual consistency, show the user their own like right away, and keep counting with operations that add up correctly in any order.
## Sources
- [Jepsen: Linearizability](https://jepsen.io/consistency/models/linearizable)
- [Werner Vogels, Eventually Consistent (All Things Distributed, 2008)](https://www.allthingsdistributed.com/2008/12/eventually_consistent.html)
- [Herlihy and Wing, Linearizability: A Correctness Condition for Concurrent Objects (ACM TOPLAS, 1990)](https://cs.brown.edu/~mph/HerlihyW90/p463-herlihy.pdf)
- [DeCandia et al., Dynamo: Amazon's Highly Available Key-value Store (SOSP 2007)](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf)
- [PostgreSQL documentation: Transaction Isolation](https://www.postgresql.org/docs/current/transaction-iso.html)
- [PostgreSQL documentation: Explicit Locking](https://www.postgresql.org/docs/current/explicit-locking.html)
- [PostgreSQL documentation: Log-Shipping Standby Servers](https://www.postgresql.org/docs/current/warm-standby.html)
- [Designing Data-Intensive Applications (book site)](https://dataintensive.net/)