0005 — Two clocks guard the reviewer's token: rank at write time, refuse at read time

• Status: Accepted
• Date: 2026-08-29
• Deciders: Gera (approved the port, decision b87972514: "Yes — port it, probing claude-sonnet-5 across all 11 slots", and authorised the override that landed the wiring). Split settled between the smith2 and ccswitch sessions the same night, each having independently measured the other's half.

The failure

agentflow-relay's review bot died on its first API call — is_error: true, 1 turn, $0, 1.7s — on every PR in the repo. Not a review that disagreed: a reviewer that never read a line of the diff. It happened twice in a row on the same PR, so it was not flaky.

The cause was one weak test in two places. The hourly rotator bucketed accounts on "did it return HTTP 200" and nothing else, and the workflow used whatever single account the rotator wrote. A live probe of the bank that night:

Eight returned 200. Four of those eight were between 80% and 96% of their weekly budget. A max_tokens:1 probe costs almost nothing, so 200 only proves the account can serve a trivial request — one at 96% answers it happily and dies partway through a real review.

The decision

Both clocks, doing different jobs. Not one or the other:

The rotator demotes rather than excludes on purpose: a pool that refuses everything leaves CI with no reviewer at all, which is worse than a bad one. The picker then declines to spend the last resort. The pool keeps one; the picker never uses it.

Why write-time ranking is not sufficient on its own

This was the live disagreement, and it was settled by measurement rather than argument. A ranked pool is a claim about the past.

On run 33231883839, slot 1 — written healthy-first at 21:35 CT — probed HTTP 401 at 22:38 CT. Sixty-three minutes, inside a single rotation window. No amount of ranking at write time survives a token that dies mid-cycle; only a read-time probe catches it. That is the whole argument for the second clock, and it does not depend on the write-time ordering being bad.

One correction belongs in the record, because a wrong version of it was argued first: anthropic-ratelimit-unified-{5h,7d}-utilization is subscription-wide, not per-model. Probed across four models on one token, the numbers are identical; an opus probe and a sonnet probe of the same bank returned the same 0.15 / 0.49 / 0.80 / 0.88 / 0.91 / 0.96. The probed model decides whether the probe is rejected — caps are model-scoped — not what headroom it reports. So the model must still match the reviewer's, for eligibility; it does not skew the ranking.

Each clock names that setting differently, and the two are easy to conflate precisely because they do the same job on different sides: the picker reads REVIEW_MODEL (pick-healthy-token.sh:42, set to claude-sonnet-5 at review.yml:192), the rotator reads PROBE_MODEL. They are not the same variable and do not have to hold the same value — but if they drift, the pool is ranked against a budget the reviewer never spends.

Consequences

• REVIEW_MODEL must track whatever model the action actually runs. A Haiku probe is not client-gated and would pass on an account whose Opus budget is gone. If a --model is ever pinned in claude_args, that line moves with it.
• The picker's script is restored from origin/main before it runs, because the checkout is PR-authored code and the next step puts eleven OAuth tokens in its environment. A guard must be able to bootstrap: the first attempt failed because git checkout FETCH_HEAD -- <path> cannot restore a file that is new in the PR introducing it. Cure was ordering (script PR first, wiring second), never a weaker guard.
• Any PR that edits review.yml is structurally unreviewable — GitHub Actions refuses to run a workflow whose file the PR modifies — and needs a manual Review Check Override dispatch, run by a person.
• Never remove the Claude Code system line or the anthropic-beta: oauth-2025-04-20 header from the probe. Without them every probe returns a generic headerless 429 and the picker reports a fully healthy roster dead. Removing that one header turns 12 of 13 tests red.

Open: this system cannot be catalogued yet, and that is a gap in the catalog

SYSTEMS.md should carry a review-gate entry — the catalog's own rule is that a system not listed does not exist. It cannot honestly carry one today.

Every signal type the relay supports reads this machine: file-age, log-match-age, launchd, self (relay/src/systems/catalog.ts:36). This gate runs in GitHub Actions, so no supported signal can observe it. And relay/tests/systems.test.ts:84 correctly refuses an active entry with an empty signals list — "a registry author forgetting the whole point".

Both halves are right, which is what makes this a gap rather than a mistake. The resolutions available were: list a local signal that proves something else and call it this one (the exact failure this whole ADR is about); mark an active system proposed to get past the test (lying about status); or leave the entry out and say why here. The third is the only one that does not put a false claim somewhere a fresh session would trust.

What would close it: a github-check signal type — the review commit status on the repo's most recent PR head being success or failure rather than absent. Absence is the failure mode, not a neutral state: a review run starved in queued contributes no check-run at all, so PR #310 sat two days reading mergeStateStatus: CLEAN, zero failing checks, never reviewed — looking better than a correctly-reviewed PR. That signal needs a token in the relay at runtime, which is an infrastructure decision and not this ADR's to make.

Until then the review gate is live, load-bearing, and invisible to the catalog.