One AI writes, another reviews: designing a multi-agent coding pipeline

An AI coding agent working alone plans, writes and judges its own work, so its mistakes have nothing to catch them. A sturdier design splits the job into parts with clear roles, the same way we split any system: a router, a writer, a reviewer, a gate and a sandbox.

Interview version: "Design a pipeline where AI agents write and ship code safely."

Prerequisites

What a coding agent does. A coding agent is a language model in a loop: it reads your files, decides on a step, runs a tool (edit a file, run a command), reads the result and repeats. Many tools work this way; this article is about how to arrange several of them, whichever you use.

Tokens and why agent tasks get expensive. A token is a small piece of text (roughly a short word or part of one) that models read and write, and most AI services bill by tokens. In loops that keep a growing conversation, later requests carry more input tokens. A 2026 study of agentic coding workloads, summarised by the Stanford Digital Economy Lab, reported about 1,000 times the tokens of its code reasoning and code chat comparison workloads, with input tokens dominating. The figure describes the workloads in that study and does not carry over as a fixed multiplier, since context compaction and caching change both usage and cost.

Queues and workers. Several agents working on one job behave like workers pulling from a queue: throughput, waiting and retries all apply. The same ideas are in why a fast server gets slow under load, why more lanes do not make a trip shorter and adding servers behind a waiting room.

Definitions

How the pipeline works

1. Route. Tasks with a narrow, checkable output (list the files that import a module, check local links, summarise a log) can go to a faster, cheaper model. Most agent tools let a sub-task run on a different, smaller model. Design decisions and the final change stay on the strongest model. Treat the split as a hypothesis: test each task type on real examples before you move it.
2. Write. The writer produces one change on its own branch, ideally in a separate git worktree so parallel agents do not overwrite each other's files.
3. Review. A reviewer reads the change against a fixed checklist (facts and numbers first, then does it do the one thing it claims, then risk) and returns issues with a severity. A reviewer from a different model family may see different things than the writer; measure whether it does (see below).
4. Gate. Pass when what matters passes: accuracy, and no unresolved high-severity issue. Otherwise the writer revises and the round counter goes up. An orchestrator, the script that runs the loop, owns the counter and enforces the cap; at the cap, a person decides.
5. Contain. Agent commands run with the least access they need. In many agent tools the shell sandbox is off until you enable it, so check yours. A typical shell sandbox, built on operating-system features such as Linux namespaces (for example bubblewrap) or the macOS sandbox, limits writes by commands and their child processes to the working folder, a temporary folder and folders you add, and sends network traffic through a proxy that only allows approved hosts. File-edit tools, MCP servers, hooks and editor extensions usually run outside it and need their own controls.

task ──> router ──(narrow, checkable)──> cheaper model ──┐
            └──(judgment)───────────> strong model ──────┴─> writer ──> reviewer ──> gate ──pass──> merge
                                                          ^                          │
                                                          └──── revise (round + 1) ──┘   cap reached ──> a person decides
shell commands (when the sandbox is enabled): writes = project + temp folders · network = allowlist proxy
file tools, MCP servers, hooks, editor extensions: separate controls (permissions, trusted installs)

Two layers protect different things. Permission rules are checked before a command runs; the sandbox is enforced by the operating system while it runs. You need both, and both kinds of isolation: network isolation alone leaves local files exposed, and file isolation alone still lets data leave. The OWASP Top 10 for LLM Applications lists excessive agency and supply-chain risk among the main risks for this reason.

Designing the reviewer

The brief limits how good a review can be, and three choices in it matter most.

A checklist, in priority order. Ours, for video scripts: accuracy (every number traceable to a measurement or a source), hook, one idea, proof, visuals, ending, house rules. Putting accuracy first means the most expensive mistakes are looked at first.

Severity with a definition. High means "would mislead a reader or break the product"; medium means "worse but not wrong"; low means polish. Without definitions, a reviewer tends to mark everything high, and the gate never opens.

A structured verdict. Ask for a table of issues and a final line that is exactly VERDICT: pass or VERDICT: revise, or better, a JSON object your script can validate. A script can easily misread free text; the code section shows how.

Fresh context. Give the reviewer the change and the brief, not the writer's whole conversation. It keeps the review cheaper and avoids the reviewer inheriting the writer's assumptions.

Worked example: our 96 review runs

We use this pattern to make the videos on this site: one AI model writes each episode script and a second model, from a different AI family, reviews it against the checklist above. The counts below come from our pipeline's database on 2026-11-04, counting stored records for the main series:

How each number was counted (one row per stored record):
  episodes           distinct episodes with a script or review        18
  script versions    script records (each rewrite is a new version)   94
  review runs        review records                                   96
  issues raised      issue entries inside all review records         715
  accuracy fail      review runs whose "accuracy" field says fail      88
  visuals fail       review runs whose "visuals" field says fail       85
  hook fail          review runs whose "hook" field says fail          34
  clean pass         review runs whose verdict says pass                2

Arithmetic:
  94 / 18  = 5.22  ->  about 5.2 versions per episode
  715 / 96 = 7.45  ->  about 7.4 issues per review run
  88 / 96  = 0.917 ->  92% of runs flagged accuracy
  2 / 96   = 0.021 ->  2% clean passes

There are more review runs (96) than versions (94) because a version can be reviewed more than once. An issue raised is something to check and may turn out not to be a defect: the same issue can repeat across rounds, and some were nitpicks. "Accuracy" in our checklist also covers timing, for example a script whose spoken length did not fit the planned 55 seconds once numbers and initialisms were counted as spoken words.

The numbers did show that a gate requiring every checklist item to pass rarely opened in our recorded runs. With only 2 clean passes in 96 runs, scripts went round and round. We changed the gate to "accuracy and hook pass, no high-severity issue left" (the hook rule is our own extra criterion for videos), and later to a person approving after review. That is why step 4 has both a rule and a cap.

How to tell if the reviewer helps

Issue counts show how busy the reviewer is and say little about its value. To decide whether a review step pays for itself, track per change:

• Validated findings: issues a person (or a test) confirmed as real. Divide by issues raised to get precision.
• Escaped defects: problems found after merge. Compare escaped defects per change on comparable workloads with and without review. A lower rate suggests benefit, but differences in task difficulty, testing and human review can also explain it.
• Shared failures: mistakes both writer and reviewer missed. A high rate suggests the two models share blind spots.
• Cost and time: tokens and minutes per change, with and without review.
• Rounds to pass: if this keeps growing, fix the gate or the writer's inputs, because more rounds will not help.

Code

A routing table can be as simple as a config file the orchestrator reads. Each task type names a model tier, the tools it may use, and whether it may write. The tier names map to whatever models you run:

# routing.yaml: which model tier and which tools each task type gets
link-check:   { tier: small, tools: [read, search],        write: false }
summarise-log: { tier: small, tools: [read],                write: false }
write-change: { tier: large, tools: [read, edit, shell],   write: true  }
review:       { tier: large, tools: [read],                write: false, family: different-from-writer }

A review gate with a second model has two parts: a review step that runs one review and reports pass, revise or error, and an orchestrator that owns the round counter. The counter has to sit outside the review step, because a step that is called again and again cannot know how many times it has run and so cannot enforce a cap on its own.

The review step. review-ai stands for your second model's command-line tool; most offer a one-shot, non-interactive mode, a read-only mode and an option to write the final answer to a file. Map the three flags below to yours. The step uses a fresh temporary folder, stops on any error, and reads only the last non-empty line:

#!/bin/sh
# review-step.sh: one review of the committed branch changes since main.
# Exit 0 = pass, 1 = revise, 3 = the review itself failed (missing diff, tool error).
set -u
work=$(mktemp -d) || exit 3
git diff main...HEAD > "$work/change.diff" || exit 3   # committed branch changes only; local edits are not included

review-ai --read-only --output "$work/review.md" \
  "Review $work/change.diff. Check facts and numbers first, then correctness, then risk.
   List issues as a table: severity (high/medium/low), file, problem, fix.
   Pass only if facts are accurate and no high-severity issue is left.
   Your last line must be exactly: VERDICT: pass   or   VERDICT: revise" || exit 3

verdict=$(awk 'NF { last = $0 } END { print last }' "$work/review.md") || exit 3
cp "$work/review.md" ./last-review.md || exit 3
case "$verdict" in
  "VERDICT: pass")   exit 0 ;;
  "VERDICT: revise") exit 1 ;;
  *)                 exit 3 ;;   # no valid verdict: treat as a failed review, not a pass
esac

The orchestrator. It runs at most three rounds, asks the writer to revise only between failed rounds, stops at once on a review error, and hands over to a person after the third rejection:

#!/bin/sh
# review-loop.sh: owns the round counter. Exit 0 = ready to merge, 2 = a person decides, 3 = review or revision error.
set -u
MAX_ROUNDS=3
round=1
while [ "$round" -le "$MAX_ROUNDS" ]; do
  ./review-step.sh
  status=$?
  case "$status" in
    0) echo "gate: pass in round $round"; exit 0 ;;
    1) echo "gate: revise (round $round of $MAX_ROUNDS)" ;;
    *) echo "gate: review failed in round $round, stopping"; exit 3 ;;
  esac
  if [ "$round" -lt "$MAX_ROUNDS" ]; then
    ./revise.sh last-review.md || { echo "gate: revision failed in round $round"; exit 3; }   # your writer agent applies the fixes and commits
  fi
  round=$((round + 1))
done
echo "gate: $MAX_ROUNDS rounds without a pass, a person decides"; exit 2

What to observe when you run it: a "pass" written anywhere except the last non-empty line does not open the gate; a crash in git diff or the reviewer ends the loop with status 3 instead of looking like "revise"; and after three rejections the loop stops and asks for a person. revise.sh stands for whatever runs your writer agent. For an automated merge, ask for JSON output and validate it with a schema instead of matching text.

Why errors get their own status. If a failed review were treated as "revise", a broken reviewer (no network, expired login, wrong path) would burn every round unnoticed and then blame the writer. If it were treated as "pass", a broken reviewer would approve everything. A separate status makes a tool failure visible and stops the loop before it wastes rounds or ships an unchecked change.

Why the counter lives outside. The same principle shows up in rate limiting and retries in general (see the Google SRE chapter on cascading failures): keep the count in one place that the limited thing does not control. Here the orchestrator centralises counting and limits the number of completed review attempts. It does not limit elapsed time: a hung review would still hang, so give each step a timeout as well.

Where a person fits. Reaching the cap means the disagreement is not converging. At that point a person reads the last review and the change, and either accepts it, fixes it, or changes the brief. In our pipeline most stuck scripts were stuck on medium issues, and the useful human fix was usually to tighten the brief rather than to rewrite the script.

Setting it up in common tools

The pattern works with any agent tool. As a concrete starting point, here is where each piece lives in two widely used command-line agents, taken from their official docs. Check the docs for your version, as options change.

Routing in Claude Code. A subagent is a Markdown file in .claude/agents/ with a model field (for example haiku for a smaller model, or inherit) and a tools list that limits what it may use (subagent docs):

---
name: link-checker
description: Checks local Markdown file links and anchors under docs/. Read-only.
model: haiku
tools: Read, Grep, Glob
---
Read every Markdown file under docs/. List each link to a local file or anchor that does not exist.
Report as a table: file, line, link, problem. Do not edit anything.

Containment in Claude Code. The sandboxed Bash tool is turned on in settings; when on, it limits writes by shell commands to the working folder, a temporary folder and folders you add, and sends network traffic through a proxy with an allowlist of hosts (sandbox docs).

A one-shot review with Codex CLI. codex exec runs a single task without a chat, -s read-only keeps the commands it runs read-only, and -o writes the final answer to a file (repository). In the review step above, that means:

codex exec -s read-only -o "$work/review.md" "Review $work/change.diff. ... Your last line must be exactly: VERDICT: pass or VERDICT: revise"

Any other pair of tools works the same way if it gives you a smaller model for narrow tasks, a read-only one-shot mode for the reviewer, an output file for the gate to read, and a sandbox you can switch on.

Failure modes and edge cases

• The reviewer and writer share a blind spot. Two models can agree on the same mistake, even from different families. Track shared failures and keep a person on anything users will see.
• The review loop never ends. Gate on what matters and let an orchestrator own the round counter, not the prompt and not the review step.
• The gate passes by accident. Matching "pass" anywhere in free text, ignoring exit codes, treating a crashed review as a result, or reusing an old review file can all let a rejected change through. The scripts above guard against each.
• A poisoned tool. Editor extensions and plugins for AI agents are software you install, and tampered builds of developer extensions do appear in marketplaces from time to time. An agent's tools are part of the attack surface. A shell sandbox does not contain an editor extension, so install from the publisher's official channel, check what changed before updating, prefer artifacts with verifiable provenance (the idea behind SLSA), and keep credentials out of places tools can read. Pinning a version helps only if the pinned version is a good one.
• An agent reaches further than it should. Enable the sandbox, deny reads of credential folders such as ~/.ssh explicitly (reads are broadly allowed by default), and keep the network allowlist short.
• Cost runs away. Long sessions carry more input. Start fresh sessions for new tasks, route narrow tasks to a cheaper model after testing, and give the reviewer only the change and the brief.

Trade-offs

Common mistakes

In an interview

1. Clarify: what do the agents change (code, docs, config), who sees the output, what happens if a bad change ships?
2. Assumptions: two model families available; every change goes through git; a person approves releases.
3. Structure: router, writer, reviewer, gate with a cap, containment.
4. Deep dive: the gate (rule, structured verdict, cap) and containment (sandbox scope, what it does not cover).
5. Trade-offs: review cost against validated findings; cheaper model speed against judgment.
6. Wrap up: metrics: validated findings, escaped defects, rounds to pass, cost per change.

Practice

Cheat sheet

Further reading

• How are AI agents spending your tokens? (Stanford Digital Economy Lab): the study of agent token use and cost variation.
• git worktree (Git documentation): separate checkouts for parallel agents.
• OWASP Top 10 for LLM Applications: excessive agency, prompt injection and supply-chain risks.
• Addressing cascading failures (Google SRE book): retry limits and why the caller should own them.
• bubblewrap: the kind of operating-system sandbox agent tools build on.
• SLSA: checking where the tools you install came from.
• Claude Code subagents and sandboxing: the routing and containment settings shown above.
• Codex CLI: one-shot runs and sandbox modes for a scripted reviewer.

This describes a general design pattern and our own measured workflow as of November 2026, not any company's private systems. Tools change quickly; check the official docs for current features.

Read next: how one agent can hand work to another when Claude Code sessions message each other, and how we write the shared steps both agents follow in one SKILL.md file.