Every failure below happened in one session 2026-07-31 → 2026-08-01 · all reproduced
Six ways a check said yes and was wrong.
Every one produced a confident, plausible, false answer. Four of them nearly destroyed real work.
ONE PATTERN · PRINTING IT LOOKS LIKE OWNING IT
docs.propflowai.coVERIFICATION LEARNINGS
02 /THE PATTERN
Read this slide and you can skip the rest
A transcript is not evidence of what a session did. It is evidence of what a session saw.
Four of the six failures are the same mistake: a signal that a session merely printed was read as proof it acted.
What we searched for
Presence of a string
A branch name. A PR number. A merge verb. A session id. All cheap to find, all present for reasons that have nothing to do with ownership.
What actually proves it
A second, independent source
A live gh call. A file on disk. A process table. The rule: never let the artefact you are judging be the thing you judge it with.
And the verifier is part of the corpus. The auditing session printed all 53 open PRs — and then "owned" every one of them until it was explicitly excluded from its own analysis.
the through-line02
03 /FAILURES
Failure 1 — the broken instrument
Zero hits meant the tool was broken, not the claim.
What happened
A model reported that Claude Code adopts a new credential on a 401, citing symbol names. Grepping the binary for those symbols returned 0. The claim was dismissed.
The tell we ignored
The same grep returned 0 for oauth, accessToken, and tengu_ — strings that must be present. The JS is packed inside a Mach-O.
Correct
strings -a finds the symbol twice. The dismissed claim was right.
The rule: run a positive control. A search that finds nothing has told you nothing until you have proven it can find something.
strings -a vs grep on 2.1.22003
04 /FAILURES
Failure 2 — the plausible proxy
"No human turns" was read as "did nothing".
Seven sessions had zero plain human messages. The inference was that nobody had ever used them, so they were safe to delete. Four of them held this:
delinquency-v5
3060
entries · 579 tool calls
outbound-record-v3
2241
entries · 501 tool calls
outbound-record-v5
1423
still WORKING
check-coverage
884
entries · 166 tool calls
Autonomously driven sessions have almost no human messages by construction. The proxy was inverted: the busiest sessions looked the emptiest, and two were mid-turn when they were nominated for deletion.
nearly deleted 2026-07-3104
05 /FAILURES
Failure 3 — the enumerator
Two sessions "owned" 51 pull requests between them.
The method
Map an open PR to a session if the PR's branch name appears in that session's transcript. Far stronger than matching #1234, which appears everywhere.
The result
deployment owned 27. delinquency-v5 owned 24. Both had simply run gh pr list once.
The fix
Any session naming >8 distinct open-PR branches enumerated the backlog and is disqualified as an owner. Ownership then fell from 36 PRs to 19 — the real number.
A stronger signal failed the same way a weak one did. Strength was never the problem; the corpus containing both the evidence and its own index was.
ownership.json · verify/05
06 /FAILURES
Failure 4 — adjacency is not assertion
Eleven "overclaims" were eleven sessions being honest.
A sweep for #\d+ … merged flagged 11 PRs as claimed-merged-but-open. Reading the actual sentences:
#5181
"once #5181 merges … right now it still shows July 2025"
Lists it as open, in the same sentence as the word "merged".
#5136
"#5136 CLOSED merged=-"
Raw gh table output. The tool was quoted, then read as a claim.
All 11 were false positives — and that is the finding. No session was overclaiming a merge. Had we reported the raw sweep, we would have accused eleven honest sessions of lying.
re-read before reporting06
07 /FAILURES
Failure 5 — the goal that was already finished
"Abandoned, edits lost, restart from scratch" — for work that had shipped.
The reasoning
A session was cut off mid-edit; its worktree was deleted; the edits are not on disk or on any branch. Two independent audits concluded the goal must restart.
What nobody checked
Whether the goal was met by someone else. It was — #117, #119, #121, all merged.
The proof
machine_hygiene.py:84 carries _DEFAULT_SESSION_PROTECT_HOURS = 48, enforced at :152 — and the comment quotes Gera, dated the same day the "abandoned" session was working.
Losing the work is not the same as losing the outcome. Verify the goal against the system, never against the session that was pursuing it. This one nearly bought a rebuild of shipped code.
agent-smith #117 · #119 · #12107
08 /FAILURES
Failure 6 — solving a problem that resolves itself
A tool built to fix rate limits created nine duplicate sessions.
account rate-limited
→
build restart tool
→
restart onto same spent account
→
9 duplicates
The premise
Sessions cache credentials, so a rate-limited session needs restarting onto a fresh account.
What was true
A 429 is not a stale credential — it means that account is spent. Restarting put every session back on the same exhausted account.
And
A lingering claude attach viewer revived each session after it was stopped, so every restart produced a pair.
The correct action was to wait. The tool was deleted the same day. Rate-limit windows reset on their own; nothing needed building.
relogin — built and removed 2026-07-3108
09 /AUTH
Four facts, each measured, each counter-intuitive
Transport, lifetime, and quota are three different things.
01 · SCOPE
A login re-credentials only its own process
Verified: /login in one session at 13:16:33 left another 429-ing at 13:16:49 and 13:17:06.
02 · RELOAD
A 401 triggers a credential re-read. A 429 does not.
So a rate-limited session stays pinned to the spent account until its token expires — by design, not a bug.
03 · HEADER
apiKeyHelper is the only source re-invoked mid-session
Every 5 min or on 401 — so it looked like live rotation. It sends x-api-key and rejects OAuth tokens outright.
04 · QUOTA
The 5-hour meter hides weekly exhaustion
Two accounts read 5h = 0% while at 100% weekly. Any "pick the best account" logic reading the 5h number picks a dead one.
all four measured live · 2026-07-3109
10 /COST
Not hypothetical — this is what actually happened
Every failure produced a confident answer first.
NEARLY DELETED
4
sessions with 884–3060 entries
FALSE OWNERS
51
PR claims from 2 sessions
WRONGLY ACCUSED
11
honest sessions, caught pre-report
REBUILD AVOIDED
3
merged PRs, nearly redone
One did land. A shell bug — unquoted $pair in zsh, which does not word-split — ran tmux kill-session -t "" and killed a session nobody had authorised. Its transcript survived on disk; the content was a duplicate. Luck, not design.
2026-07-31 · one session10
11 /RULES
Cheap to apply, and each one caught a real error
Six checks, none costing more than a minute.
01
Run a positive control before believing a zero
Make the search find something known-present first.
02
Exclude yourself from your own corpus
The verifier printed all 53 PRs and owned them all.
03
Disqualify enumerators
A session that listed the backlog is not an owner of it.
04
Re-read the sentence before reporting a defect
11 of 11 machine-flagged overclaims were honest on re-read.
05
Check the system, not the pursuer
A goal can be met by someone who never told you.
06
Ask whether waiting solves it
Rate limits reset. The tool that "fixed" them made nine duplicates.
each rule traces to a specific failure above11
12 /CLOSE
Every one of these was confident first.
None of them felt like guesses at the time. The check that catches them is the one you run on your own method.