Build a verifier that doesn't trust the builder

Every child in the last lesson finished with some version of “done - re-extracted, index updated.” You read that, moved on, and closed the backlog ticket. Here’s the spoiler, stated as a claim this lesson re-derives: batch 07 claimed all 12 of its URLs were clean. Only 9 of those 12 claims actually held up when a second agent went and checked. The other 3 split two different ways, and which way matters more than the count does.

That’s the last slice of this module’s villain: one window cannot hold the job, isolation buys you the room to split it - and the handoff, the report one window hands back to the window that dispatched it, is exactly where the tax comes due. Notice what actually happened in the last lesson: you accepted a claim from the one party in this whole exchange with every incentive to report success - the agent that just did the work. stash’s extractor still mangles some sites even after your fix; the full-text search still drops accented characters sometimes. A child that ran stash reextract and got a clean exit code has no way of knowing whether the text it just indexed is actually readable, or whether “café” made it into the search index as “caf.” It reported what it did. It didn’t verify what it produced.

That’s a structural problem, not a prompting one - no amount of “please double-check your work” fixes an agent checking its own homework. The fix is a second, independent agent, spawned separately, whose only job is to distrust the first one’s report on principle and go look for itself.

Spawn the verifier against the builder’s own transcript, not its summary

The verifier doesn’t have to take the builder’s word for anything, because a Pi session persists its full, uncompacted history to a JSONL file on disk, regardless of what got summarized into context along the way. One catch carried over from the last two lessons: those children ran with --no-session, so they left no file behind - spawn any child whose work you intend to verify with --session ~/.pi/agent/sessions/reextract-batch-07.jsonl instead. Point the verifier at that file rather than at the builder’s closing message, and it isn’t reading a summary - it’s reading exactly what happened, tool call by tool call:

pi --mode json -p "Read the builder's session log at
  ~/.pi/agent/sessions/reextract-batch-07.jsonl. It claims to have re-extracted
  12 URLs and updated their search index entries. Do not trust that
  report. For each URL: (1) read the actual stored article text and
  confirm it contains no nav/ad boilerplate, (2) query the search index
  directly and confirm the entry changed and preserves accented
  characters where the source had them. Report per-claim: verified,
  failed, or could-not-verify - with the specific mismatch for anything
  that fails." \
  --no-session \
  --tools read,bash \
  --model openai/gpt-4o

Before you run that, predict the split. The prompt asks for three verdicts, per URL, across all 12: verified, failed, or could-not-verify. Given what you already know - the extractor still mangles some sites, search still drops accents sometimes - guess roughly how 12 real claims split across those three buckets. A reasonable guess: most hold up, a couple don’t, maybe one stays genuinely unresolved.

Here’s what the verifier actually reports back, per URL (a declared toy split, chosen to make this walkthrough checkable - see the numbers note in the last lesson):

Verdict	Count	What it found
Verified	9	stored text clean, index entry matches, accents intact
Failed	2	post-142’s index entry still reads “caf” where the source has “café”; a second URL still carries nav boilerplate the extractor missed
Could-not-verify	1	the search index query for that URL returned inconsistent results across two attempts - not proof of failure, and not proof of success either

Nine of twelve. Not zero, and not twelve. That’s the number this lesson opened on, and it’s exactly the mix that makes “please double-check your work” the wrong fix: the builder didn’t lie about the other three, it was simply never in a position to know.

Two choices here matter more than they look. First, the toolset: read,bash only - no write, no edit. The verifier’s whole job is to look, not to touch; giving it a narrower surface than the builder is the same instinct as the damage-control gate from an earlier module, just applied by omission instead of by rule. Second, a different model than the builder used. A verifier that shares the builder’s blind spots will miss exactly what the builder missed; a genuinely independent second opinion is worth more when it isn’t the same model checking its own reasoning twice.

Decompose “done” into claims that can actually be checked

The instruction above already does the important part: it doesn’t ask “did this work?” - it asks the verifier to break “done” into pieces, each one either true or false against something real. “Re-extracted” isn’t one claim, it’s at least two: the stored text changed, and the new text is actually clean. “Index updated” is a separate claim from either of those. That decomposition is exactly what turned one vague report into the 9 / 2 / 1 table above - three checkable pieces per URL instead of one trusted sentence.

“Done” is never one claim. Decompose it until every piece is checkable, or the verifier is checking nothing.

That decomposition belongs in a durable file, not a one-off prompt - put it in the verifier’s own system prompt (.pi/agents/verifier.md, or handed in via --append-system-prompt), the same way .pi/APPEND_SYSTEM.md captures stash’s conventions so you stop re-typing them every session. And that’s not incidental - it’s the whole reason this pattern is worth building at all.

Why “no chat input” is the point, not a limitation

Look again at the flag you already know from the last two lessons: -p. Print mode. One prompt, one run, exit. You’ve been using it because it’s the natural way to spawn a headless child - but for the verifier specifically, that constraint does something you haven’t needed yet: it removes your ability to argue with it.

Sit with what that means in practice. If the verifier wrongly flags a clean re-extraction as broken, or worse, waves through one that’s actually still full of nav cruft - you can’t just reply “no, that one’s fine, ignore it” and move on. There’s no follow-up turn to send it. The only lever you have is the file: open .pi/agents/verifier.md and change what it checks, or how. That’s a real constraint, and it’s a good one. It’s the exact same discipline behind writing .pi/APPEND_SYSTEM.md instead of re-explaining stash’s conventions in every session - except here it isn’t a convenience, it’s enforced. A chat-shaped reviewer lets you patch a bad judgment call in the moment and forget it by next week.

No chat input means every fix becomes a durable rule the first time - there’s no other way to reach this verifier at all.

Here’s that same shape, made concrete. Nine chores went out overnight; only the worker’s own report comes back unless you’ve wired something in to check it. Before you touch anything, guess: with nothing wired in, how many of the nine do you think actually got done correctly, versus just reported done? Watch the night play out, then choose what gets wired into the re-run - and compare what the morning report claims against what actually happened:

Nine chores, queued for an agent to run overnight. The terminal replays the night as it happened; the panel is the morning report - the only thing you’ll actually read. Watch what its checkmarks are made of, and when playback pauses, decide what the run has to get past before you believe it.

overnight batch - ledger-app

overnight batch · repo: ledger-app · 9 chores queued · nobody watching

the morning report

The flaw rate here is theatrical - seven duds in nine - because the point is what each wiring would have caught, not the odds. The checks wear different clothes per tool (a CI step, a test command the loop must pass, a verifier agent), but the question is always the same: what does this run have to get past before “done” reaches you? And the residual is real: no check catches a requirement nobody wrote down.

Notice the ceiling even at full wiring. A verifier that checks the code compiles, the tests pass, and the full path runs still can’t catch the one failure mode nobody encoded as a check - because a verifier can only verify a claim somebody already wrote down as falsifiable. That’s not an argument against building one. It’s the argument for building the right claims into it, and for treating “what should this verifier check” as a question that gets revisited every time something slips through, not answered once and forgotten.

One more ceiling, and this one doesn’t get solved in this lesson: nothing verified the verifier. It ran with read,bash and a prompt you wrote, and you read its 9 / 2 / 1 report the same way you read the builder’s - on trust. Chase that far enough and you’re proposing a third agent to check the second, then a fourth to check the third, and the regress never resolves, it just moves. This module doesn’t answer it. The next one is where more agents means more of exactly this problem, multiplied by however many of them you add - pick it up there.

What you’ve actually built

Two agents, neither one trusting the other by default. A builder that does the work and reports on itself, and a verifier that reads the builder’s real transcript, breaks its claims into pieces small enough to check, and can only ever push back by having its rules edited - never by being talked out of a finding. That’s a genuinely different relationship between agents than “one delegates to another.” It’s closer to the relationship you’d want between a contributor and a reviewer who’s never met them: assume nothing, check everything falsifiable, and write down what you learn so the next review doesn’t have to relearn it.

If you’ve used Claude Code’s Task tool, or something like it, this is what’s running underneath - minus the parts it hides from you. Task gives you retries, a timeout, cost accounting rolled into your own usage, and a scoped subagent role picked from a list, for free, out of the box. The naked spawn gives you none of that by default; every one of those is a decision you make in your own driver, or a gap you leave open. That’s the actual trade of building this yourself instead of trusting a dispatcher tool: less magic, and a harness where every piece of what “delegate to a subagent” means is something you chose, not something a black box decided for you.

Replay the ledger with what this module added:

Layer	What it costs	What it buys	Who decided
A spawned child	Tokens spent in a separate window, never counted against yours	Removes the failure of one job filling the only window you have	You
A child’s session file	Disk, not context - persists whether or not you ever read it	Removes the failure of trusting a summary instead of the real, tool-call-by-tool-call record	You
An independent verifier	A second full agent run, its own tokens, its own window, its own model choice	Removes the failure of an agent grading its own homework	You

Three new rows, and the same column that’s carried every row since the first one: you decided. Not Pi, not the builder, not the verifier - the spawn call, the --session flag, the second model, all of it something you chose to spend.

Isolation works. Two agents just did real work on stash, in separate windows, and neither one had to hold the whole job at once. Now you’re paying for the handoff between the isolated parts: the report a child hands back, the transcript a verifier has to go read instead of trust, the regress of who checks the checker that this module leaves open on purpose. The next scale-up isn’t more of the same kind of pair - it’s a structure: someone who plans, several someones who execute, and a rule about who’s allowed to touch code directly. Next: build a small team, and learn honestly when it’s not worth the overhead.