How thousands of writers update one repository: objects first, refs last

Many writers can upload immutable Git data in parallel, then publish changes through small compare-and-swap transactions on branch pointers. Writers using separate branches avoid most pointer conflicts; writers sharing one branch still contend.

Prerequisites

Git stores history as objects (commits, trees, file contents) and names each branch with a ref that holds one commit ID. Optimistic concurrency skips locks and checks for a clash only at write time, the version check from two fans, one seat.

Definitions

How does one repository take many writers?

A push carries two kinds of data. Git identifies an object by hashing its type, length and contents, so identical objects share an ID; hash collisions need separate handling. According to GitHub's write-up, pack uploads use distinct immutable storage keys, so writers upload independently. A ref such as refs/heads/main is a few dozen mutable bytes that every writer on that branch wants to change, which makes it the hot key.

Publication order. A writer uploads its pack first and publishes the ref last, so a committed ref never points at an unfinished upload. Connectivity checks and garbage collection must keep every object reachable from a published ref.

Conflict checking. The publish step is one small transaction, "move main from A to B". If main no longer equals A, the store rejects it; the writer reads the new value, rebases or merges, and retries. Git does this locally: git update-ref refs/heads/main NEW OLD refuses to change the ref unless it still holds OLD, and --stdin mode applies several such checks all or nothing (Git docs).

Durability. The ref store must survive machine loss without forgetting a published ref. In Raft, a leader commits a current-term entry after durable replication to a majority, and election and log rules preserve committed entries across leadership changes.

What GitHub's write-up reports

This section follows GitHub's November 5, 2026 post, Inside GitHub's new Git storage architecture; an earlier post covers why the change was needed.

On a push, a replica uploads the pack, then commits one Ref-CAS transaction that adds it to the manifest and moves the refs, checking each expected value and locking only those refs. A transaction commits once a majority of log replicas record it. The post calls the LSN of the latest commit the repository's version; this article calls it the freshness token. A read that already holds it gets a reply with no references.

The first prototype used a write-ahead log for reference state, indexed by one conditionally updated blob, and reached only the mid-hundreds of writes per second. Splitting the index raised write throughput, but every read then consulted every segment. In internal testing the new design handled 5,000 concurrent agents sending 7,200 pushes per second to one repository. Separately, it notes that agents mostly push to their own branches, so most updates do not contend. The write-up puts Ref-CAS in the tens of thousands of operations per second and says it is in limited private preview.

Worked example

Assume a 10 ms round trip. The serialized line counts index updates that each wait for the last; with batching one update can carry several pushes.

Separate refs, all parallel: read + CAS = 2 round trips      ≈ 20 ms total
Serialized index updates:    1 update per RTT = 1 / 0.010 s  = 100 updates/s
                             1,000 updates                   ≥ 10 s
100 writers on one ref, synchronized rounds:
  attempts   = 100 + 99 + ... + 1 = 100 × 101 / 2           = 5,050
  rejections = 5,050 - 100                                  = 4,950

Code: what contention costs

This in-process simulation gives the ref store an assumed 10 ms round trip. Each agent reads its ref, attempts a CAS, and retries on rejection. The script and both recorded runs are kept with our lab files (labs/thousand-agents-push-one-repo); run it with python3 -I ref_cas_lab.py.

# ref_cas_lab.py: many writers, one ref store, compare-and-swap
import asyncio, time

RTT = 0.010  # assumed 10 ms round trip to the ref store

class RefStore:
    def __init__(self, one_at_a_time=False):
        self.refs, self.ok, self.rejected = {}, 0, 0
        self.gate = asyncio.Lock() if one_at_a_time else None  # serialized model

    async def read(self, ref):
        await asyncio.sleep(RTT)
        return self.refs.get(ref)

    async def cas(self, ref, expected, new):
        if self.gate:
            async with self.gate:
                return await self._cas(ref, expected, new)
        return await self._cas(ref, expected, new)

    async def _cas(self, ref, expected, new):
        await asyncio.sleep(RTT / 2)          # request travels
        ok = self.refs.get(ref) == expected   # the compare
        if ok:
            self.refs[ref] = new              # the swap
            self.ok += 1
        else:
            self.rejected += 1
        await asyncio.sleep(RTT / 2)          # reply travels
        return ok

async def agent(store, i, ref):
    while True:                               # read, build on top, try to swap
        old = await store.read(ref)
        if await store.cas(ref, old, f"c{i}-on-{old}"):
            return

async def run(label, agents, same_ref, one_at_a_time=False):
    store = RefStore(one_at_a_time)
    t = time.perf_counter()
    await asyncio.gather(*(agent(store, i, "main" if same_ref else f"agent-{i}")
                           for i in range(agents)))
    secs = time.perf_counter() - t
    print(f"{label:<42} ok={store.ok} rejected={store.rejected} time={secs:.2f}s")

async def main():
    await run("1,000 agents, own refs, parallel", 1000, False)
    await run("1,000 agents, own refs, serialized", 1000, False, True)
    await run("100 agents, all on main, parallel", 100, True)

asyncio.run(main())

Two runs on 2026-11-06, Mac, Python 3.14, standard library only:

Counts matched in both runs; times depend on event-loop scheduling. The code has no round barrier; 4,950 rejections occurred in these runs because every remaining writer read after the same sleep. The triangular formula assumes synchronized rounds. These timings measure the simulation, not Git, storage or consensus.

Failure modes and edge cases

Lost majority. If the log cannot reach a quorum, it cannot commit, so pushes must fail or wait instead of publishing on a minority. Server failover, active-passive vs active-active covers the failover side, including fencing.

Ambiguous commit. After a timeout, the outcome is unknown. A fresh read showing the proposed value confirms the intended state. A different value cannot establish failure because another update may have followed. General services need a durable operation ID or transaction-status lookup before retrying an operation that must run once.

Orphaned uploads. A writer that uploads a pack and dies leaves a pack outside the manifest. Cleanup must coordinate with publication so a paused writer cannot publish a pack after deletion. One general design uses expiring upload leases, rejects publication after expiry, and retains retired packs while readers may still need them. GitHub's write-up leaves its detailed GC safeguards for another post.

Stale reads. A stale ref still names a valid older snapshot; reads that need the latest state use the freshness check, and stale likes and safe withdrawals covers when staleness is fine. Contention shows up first in rejected CAS calls per ref and p99 push-to-commit time.

Trade-offs

My take: the split is old and well understood, and the hard part is operating it. You run a consensus log, keep pack counts flat with compaction, and delete data safely. A busy shared main still serializes and needs a merge queue or batching.

Common mistakes

In an interview

Clarify how many writers share one record, assume a round trip, split payload store from pointer store, go deep on CAS and the quorum log, weigh the hot key, and close on the metrics you would watch.

Practice

Cheat sheet

Further reading

• GitHub, Inside GitHub's new Git storage architecture: the original write-up with the Ref-CAS design, rejected prototypes and test numbers.
• GitHub, Building Git infrastructure for agent-scale development: the earlier post on why the old design struggled with more writes.
• Git, git-update-ref: CAS and all-or-nothing ref transactions you can run locally.
• Microsoft Learn, Blob conditional headers: If-Match and ETag preconditions on one object.
• Raft paper: leader election, log replication and the rules that keep committed entries.

This article explains general patterns for write contention on a shared record. Statements about GitHub's system are limited to what their public write-up states; the lab and arithmetic are our own illustrations, not measurements of any company's systems.