Build Fast; Review Faster

AI code is already the majority of merges at the places that ship fastest (Anthropic). The bottleneck is not coding; it is review & dispatch.

I think unit of trust is the diff + CI signal, never a ten-page CLI response. This post is the architecture for moving away from manual review to a headless, evidence-gated pipeline. Build L2 this week.

Any process that requires a human to read a multi-page agent report every cycle will not scale. It does not matter how good the model is. The human becomes the serial bottleneck, context drifts, and the system stays in chat theater. The correct unit of work is a machine-attested artifact that can be gated automatically.

Climbing the ladder:

L0 — Exhaustive human read

Does not scale. Out dated since early 2026.

L1 — Disciplined manual review

Prompts + Skills, structured output, human still in the loop for every cycle. Good for small projects (e.g., vibe coding projects, or less than 200K LOCs)

L2 — Headless review service

The reviewer stops being a chat window & becomes an API call.

• Input: agent report + the original primer as system prompt.
• Output: structured JSON only — claim ledger, risks, decisions_needed, next runbook.
• Gate:
  • decisions_needed empty and risk below threshold → auto-dispatch the runbook into a fresh headless Claude Code session.
  • otherwise → post only the decision items to a human queue (Slack/Linear card).

Seniors answer three questions. State lives in ledger.json in the repo, not in any context window. Every review is a fresh, stateless call by construction. Continuation-prompt drift is structurally eliminated.

Claude Code already ships the primitives: claude -p --output-format json --bare. The wrapper is ~300 lines: session-end hook → reviewer call → parse → gate → either spawn or queue.

L3 — Evidence mode

Same wrapper, but the reviewer now receives machine-pulled evidence alongside the report: CI results, test counts, diffstat, lint output, coverage delta. The verdict ceiling rises from SUPPORTED to VERIFIED for any claim the machines can check. Reports shrink to open questions and anomalies. The “results” half is attested; only the “questions” half reaches humans. This is the natural generalization of a runbook-review gate.

L4 — Supervised autonomy

The orchestrator itself is an agent. Humans set objectives, work the decision queue, and audit by sampling. QA by sampling, not exhaustive inspection. Same model used in manufacturing once process capability is high enough.

Implementation constraints (non-negotiable)

• Cap cycles per objective. Force a human checkpoint every N cycles. Destructive operations stay hard-gated behind decisions_needed. Auto-dispatch without these limits produces runaway loops.
• Two models agreeing is not truth. CI is the only arbiter. That is why L3 is the priority immediately after L2, not a later optimization.
• Gate auto-dispatch per repository on CI maturity. A repo without a real test suite stays at L1 until it earns one.

Instrumentation

Track four numbers:

• claim-survival rate (% of SUPPORTED/ASSERTED claims later falsified by CI or Section 0)
• human touches per cycle
• decision-queue latency
• senior-hours per shipped feature

These numbers decide when a repository is allowed to move up a level and when the reviewer itself has drifted.

Personal TL;DR

AI coding = AI Review