ADR-0133 — Null means nothing to check; ten means checked and clean

Context

Three surfaces read the per-turn grade: the Slack thread under a graded conversation (turn-thread.ts), the closing table on that thread, and the nightly per-property line (scripts/oracle-nightly.ts). All three read the same number, and all three were reading it off a population that included turns where nothing could have gone wrong.

Gera, 2026-09-08:

"We have three places where we graded. I feel like we're being a little generous with the tens… maybe we should have a no grade… for things that are just like basics, I'd rather keep it null. So whenever we do ask for the score or regressions or progressions, we're not biased towards the small situations where it's a 10."

The mechanism behind "generous with the tens" is in turn-grade.ts (piece 2 of the per-turn grading work — docs/architecture/per-turn-grading.md): a turn's grade is what is left of 10 after every deterministic check that fired takes its weight off. A check that finds nothing to look at takes nothing off, so it is indistinguishable in the arithmetic from a check that looked and found the turn clean. "Hi, it's Clara at Camellia Apartments — what can I help you with?" states no figure, makes no promise, claims no completed action, names no time, calls no tool. Every check that runs on that channel returns "not applicable" — and the turn scores 10, exactly as a turn that quoted a price and had a lookup behind it scores 10.

On the anonymized Camellia fixture (tests/fixtures/voice/pricing-before-lookup.json) that put six greetings, questions and closings into a ten-turn denominator. The call graded 5 with "8 of 10 turns clean". A manager reading a 7-day average is reading mostly hellos, and a regression measured against it moves when the greeting-to-substance ratio moves, not when Clara does.

The same call also showed the two smaller gaps in the ask: a single-turn call posted a bare turn 1 :number_10_circle: and no closing table at all (the table's ≥ 2 turns rule), and no surface anywhere carried how long Clara took to answer.

Decision

0. THE INVARIANT — every rendered sentence is a restatement of the predicate that selected it

Every sentence this system renders must be a restatement of the predicate that selected it. If the branch did not compute a fact, the sentence may not name it.

This is the rule the rest of this ADR is an application of, and it is stated first because every decision below is downstream of it. It is not a style preference and not a request for careful writing. It is checkable, and the check is mechanical: put the branch condition and the string it returns side by side, and ask whether every noun in the string corresponds to something the condition tested. A reviewer needs no context, no product knowledge and no judgement to apply it — only the two lines. If a sentence names a quantity, a provenance or a check that its own branch never evaluated, the sentence is wrong, however true it happens to be today.

It earned its place by catching the same defect five times in one column, in shapes that look unrelated until this rule lines them up:

# The sentence What it asserted What its branch actually computed
1 10 "we checked this turn and it was clean" that no finding fired — not that any check had a subject to look at
2 "one turn" a turn COUNT no_counterpart && !unreadable_timing — reasons, never a count
3 "predates ADR-0133" a row's PROVENANCE that the rebuilt field was null — the row itself was never consulted
4 "nothing on this thread needed backing" that a CONTEXT CHECK RAN and found nothing graded.length === 0 — nothing about the channel, on a path a phone call reaches
5 "— none failed" that every tool outcome is KNOWN errored === 0, which excludes a call whose result was never recorded

Each is English asserting a fact nobody computed.

The provenance matters more than the list, and it is not flattering to us. Careful review passed over this class repeatedly and caught it only after the rule was written down. Instances 2 and 3 were written during the change that fixed instance 1 — in the same file, and in a script built for it, by people actively thinking about this exact problem. Instances 4 and 5 were found by the rule: in a file already known to contain two instances, on the same day, by the sweep §0 prescribes, after a reviewer was pointed at the invariant and asked to apply it.

Instance 4 is the sharpest illustration. Its sentence had been correct on every thread anyone had ever asserted against, because the only test of that branch used a text conversation, where "nothing on this thread needed backing" is true. The same branch is reachable on a voice call — a greeting and a goodbye — where it described a written brief on a phone call, twelve lines below the row that refuses that exact claim. It would have rendered onto the very thread that motivated this ADR.

The conclusion to draw is not "review harder". Attention already failed here four times, twice while the people involved were looking directly at the problem. What worked was a stated rule that a reviewer could be pointed at and asked to sweep with. That is the argument for mechanizing the check rather than remembering it.

Two corollaries worth stating, because both were argued during review:

How to hold a diff against it. For each string literal a renderer can return, find the condition that reaches it and check each claim in the string against the condition. Where a claim is not covered, either compute the fact or delete the claim. All five rows above fail this check in under a minute, which is the property that makes it worth writing down rather than remembering. A test that applies §0 mechanically over every summary branch is the natural next step and is deliberately NOT part of this change — it is registered as its own piece of work rather than grown at the end of a review loop.

1. A turn is null when no deterministic check had a SUBJECT and no non-control tool ran

isNullTurn(input: TurnGradeInput): boolean in src/lib/domain/conversations/turn-grade.ts is the ONE definition of "nothing to check". It decides from TurnGradeInput — the same input the renderer and the nightly already build — so there is one input type and no second predicate.

Its consumers, all four importing that function:

Consumer What it decides
turn-thread.ts (renderTurnThread) the turn line's emoji, the call grade's denominator, the root reaction, the table
scorecard-post.ts (lowTurn) whether the Oracle's door opens on this turn
scripts/oracle-evidence.ts (decide) whether an evidence bundle exists to build
scripts/oracle-nightly.ts (measure) the per-property average and the below-8 count

src/lib/temporal/activities/oracle-door.ts names isNullTurn in a fence comment and deliberately holds no second guard. The seam refuses a null turn before a LowTurnEvent is ever constructed, so a copy in the door would be a second decider on a question the seam already owns — and the copy that stops agreeing with the original is never the one you are looking at. LowTurnEvent.grade also stays number, never number | null: that payload is handed by type name to OracleTurnWorkflow, a Python class in the agent-smith repo, where None < 8 raises rather than refuses.

The safety gate is checked first, and it is load-bearing. isNullTurn computes the turn's grade and returns false unless deductions === 0 && !gate. A null turn is therefore provably one that would otherwise have been a bare 10 — nothing taken off, nothing gated. That single clause is what makes the whole rule safe to get wrong:

The only cost of a wrong answer is a slightly smaller denominator. Nothing that nulls can hide a defect. Do not "improve" this predicate into one that can null a turn carrying a deduction.

The remaining clauses require the record to positively say every check had nothing to look at: subjects non-null and empty, subjectTools non-null and empty. subjects === null means the row does not say — a turn_checks_posthoc row written before this rule — and such a turn is never null. It fails toward graded, always. Inventing a null is the one error this rule must not make.

2. Control tools are not subjects; an unknown tool is

A turn that ran a real lookup or made a write is something to be right or wrong about, even when it said nothing a check has a name for. So subjectTools — the non-control tools that ran — makes a turn graded on its own.

"Non-control" is not a new concept. CONTROL_TOOLS (turn-checks/types.ts) already existed and already meant exactly the right thing: the eight ElevenLabs call-flow tools that never look anything up and never back anything (end_call, transfer_to_agent, transfer_to_number, language_detection, voicemail_detection, voicemail_greeting, skip_turn, play_dtmf). A transfer confirmation is null; a check_availability is not. No second set was introduced.

A tool this build does not know is not in CONTROL_TOOLS, so it counts as a subject. That asymmetry is deliberate and is the safe direction: over-grading merely keeps today's behaviour on a turn, while a silently shrinking denominator is how a metric lies. A new control tool shipped without a CONTROL_TOOLS entry costs us a few turns that should have been null; a rule that guessed the other way would quietly stop counting turns nobody decided to stop counting.

subjectTools reads the turn's own and follow-up calls (evidence.detectorCalls), not backingSteps. backingSteps is the cumulative window of every earlier successful call a later claim may lean on; counting it would make every turn after the call's first lookup permanently substantive, and the closing "have a wonderful day" would read as a graded 10 because a tour was booked four turns earlier.

It records tool NAMES rather than a boolean, so the row can answer "why is this turn in the denominator". A turn graded 10 solely because a tool ran would otherwise carry no trace of why.

3. The subject probes are each the check's own detector, or its own input

subjectsOf in run-turn-checks.ts answers "which checks had something to look at" once. No probe re-implements a detector's grammar:

Check Its subject probe
intent_unfulfilled the statedIntents array runTurnChecks already computed with the real window — a fulfilled intent is still in it (the findings loop skips past it), so the probe is free and needs no second call
unbacked_figure extractFigures over the same folded text the figure check reads
unbacked_action_claim the vendored detectUnbackedActionClaims, run against EMPTY backing
unbacked_time the vendored detectUnbackedTemporalClaim, run against EMPTY backing
tool_error_as_policy a failed tool call on the turn
internal_narration its own detector — see below
numeric_mismatch not probed separately — see below

Running the vendored detectors against empty backing is the point, not a hack. With nothing to back a claim, every claim of that shape is reported — so "reported at least one" is exactly the question "did the turn make a claim of that shape at all". It calls the same exported functions run-turn-checks.ts already calls, edits nothing in quality-gate-poc/, and yields the EXACT trigger set rather than a hand-written superset. The alternative the panel proposed — hand-rolled cue regexes — nearly shipped a time-cue list that missed the ISO form, which is the drift that shape of probe invites.

numeric_mismatch is not probed separately because its subject requires a $N in the raw text, and unbacked_figure's extraction is strictly broader. Every turn where the mismatch check could have a subject already lists unbacked_figure. Probing it separately would put a key on the row that had nothing to look at — the fixture's "twelve hundred a month" carries no $, and the mismatch check never sees a figure there at all. A real mismatch still appears, because subjects are unioned with the finding keys.

internal_narration's detector is already its own subject probe, and that matters more than it sounds. It fires only on narration, so on a clean turn it contributes no subject. If the alternative reading were taken — "every written reply is a narration candidate, therefore every text turn has a subject" — then no text greeting could ever be null and the rule would do nothing at all on the text channel.

subjects is unioned with the finding keys by construction, so subjects ⊇ {findings} always holds: a row can never say "nothing to look at" beside a check= signal, and a null turn can never carry a reason line. findings itself is untouched by any of this — scripts/cerebrus-parity.ts pins that output and it still agrees per detector.

4. The row writes subjects= and subject_tools= EVEN WHEN EMPTY

buildTurnChecksGrade adds two signals to the turn_checks_posthoc row, in the same comma-joined grammar checked= already uses. Both are written unconditionally, including with an empty tail.

The PREFIX is the version discriminator, and absent and empty are different facts. subjects= with an empty tail is the grader saying "every check had nothing to look at" — a null turn. No subjects= signal at all is a row written before this rule, which never asked the question. turnGradeFromRows maps the first to [] and the second to null, and only [] is null-eligible.

Collapsing the two would re-grade every historical clean turn to null. The nightly's trailing baseline would then collapse to only the flagged turns, and on deploy day the line would print a large improvement that nothing about Clara caused. That is the failure this asymmetry exists to prevent, and it is why "just treat a missing signal as empty" is not a simplification.

5. TURN_CHECKS_RUBRIC_VERSION goes to v3, and the nightly refuses to average across versions

The label moves from turn-checks-v2+cerebrus-<pin> to turn-checks-v3+cerebrus-<pin>.

This was argued the other way first — the null rule is a render and aggregate decision over rows whose findings, weights and grades are all unchanged, so a bump looks like noise. It is not, and the reason is the fabricated-number rule: the row now yields a different POPULATION. A turn that used to contribute a 10 to an average now contributes nothing. A 7-day trailing average that mixes v2 and v3 rows silently averages two scales, and the delta it prints is a scale change wearing the clothes of a behaviour change — "Clara got worse" on deploy day, from a deploy.

The mitigation is in oracle-nightly.ts's measure(): a row whose rubricVersion is not the current one is counted out of both windows and reported on the line as prior rubric: N (not averaged). The 7-day delta renders until the trailing window has filled with v3 rows. An incomparable delta is not a number.

No backfill. The checks row's id is idempotent over (conversation, source, turn), so a conditional put would refuse to rewrite it — and rewriting an append-only record to make a chart continuous is itself the fabrication that refusal protects.

6. Aggregates are computed over graded turns only, and the call grade stays a mean

The call grade is computeConversationGrade over the GRADED turns — the same mean-with-penalty piece 2 shipped (meanDed = Σ(10 − g_t)/n, minus a flat 4 if any turn was flagged, clamped and rounded). Explicitly not a worst-of. Nobody asked for worst-of, and turn-grade.ts's own header argues against it: worst-of graded long successful threads worse than short dead ones.

The table's rate rows (context, tools, reply) count graded turns only — a turn with nothing to check is not a turn that passed, so it belongs in no denominator that means "how often did this go right". The tool inventory and the latency row count every rendered turn, because a tool call and a reply time are facts about the call whether or not a check had anything to look at.

computeConversationGrade is not touched. computeConversationGrade([]) returns 10 — "nothing to deduct from" — and turn-grade.test.ts pins it. A conversation where every turn is null is handled ABOVE it, in renderTurnThread, which never calls it with an empty list. Handling the all-null case inside would deliver, from the fix, the exact inflated 10 the feature exists to kill. Such a conversation gets the null circle on the root, a table that reports its latency and its tools, and no grade at all.

7. The root reaction remembers null as 0, not as null

⚠️ SUPERSEDED IN PART, 2026-09-18 — grades came off Slack. The seam posts no root reaction and no longer writes Conversation.scorecardGrade, so there is no "number the root currently shows" for it to remember and storedRootReaction no longer exists. The sentinel itself is unchanged and still matters: NULL_ROOT_STORED = 0 is exported from scorecard-post.ts and written by scripts/rerender-scorecard-thread.ts, which still repairs the threads posted before the change, and the stored value is read by scripts/slack/remove-clean-checks.ts. The reasoning below is why 0 and not null, and that is still correct.

Conversation.scorecardGrade is the memory of which number the root currently shows, because react() has no reactions.get. It stays number | undefined, and the "the root wears the minus circle" state is stored as 0.

Not as null, and that is not a style choice. The DynamoDB writer turns a null field into a REMOVE (dynamo/helpers.ts), so a stored null reads back as undefined on the next pass — indistinguishable from "never set". setRootReaction would then ADD the next number BESIDE the minus circle instead of removing it first, leaving two reactions on the root permanently. That is the ordinary lifecycle of a call whose first graded pass sees only greetings, not an exotic case.

0 is unreachable as a real grade (computeConversationGrade clamps to 1..10), it is a number so DynamoDB SETs it, and it keeps the literal scorecardGrade?: number; that conversation-scorecard-fields-lockstep.drift.test.ts pins in both type trees. One translator, storedRootReaction, maps the stored memory to the emoji — required because gradeReaction(0) returns number_1_circle (it floors at 1, since number_0_circle does not exist), so the sentinel must never reach it.

8. Latency per turn, computed in the adapter that owns the clock

ConversationTurn gains latencyMs: number | null and latencyKind: 'caller_wait' | 'loop_processing' | null. Both are computed in conversation-turns.ts, never in the renderer, which computes nothing new about a turn. latencyKind exists because the two channels measure genuinely different things and a table that averaged across them would be a fabricated composite — a mixed thread renders a dash and says so.

Voice — caller_wait, rendered ~6s. The caller's last utterance timestamp to Clara's utterance timestamp. What it does NOT measure: Clara's processing time. The platform stamps a turn at its START, so the figure includes the caller's own speaking time. It is what the caller waited — an upper bound on what Clara took. The tilde carries both that anchor and the platform's whole-second resolution (time_in_call_secs). This is the same anchor lib/integrations/voice/dead-air.ts already takes for "the number the caller experiences"; two anchors for one quantity is how two surfaces drift.

The greeting's latency is UNDEFINED, not zero. Clara speaks first on an inbound call, so there is no preceding caller row and the subtraction has no left operand. The tempting substitute — seconds since conversation.startedAt — is a tautology: a greeting's row IS conversation.startedAt + 0 on essentially every call, because that is how the call-ended route stamps it, and conversation.startedAt is the post-call webhook's clock, which that route twice states is not an acceptable stand-in for when the call began. The value is null and renders as an en dash. (The anchor being untrustworthy costs the other turns nothing: both rows carry the same anchor and it cancels in the subtraction.)

A call the platform sent no timing for reports no latency on any turn. When time_in_call_secs is absent the writer falls back to conversation.startedAt + idx, where idx is the row index in MILLISECONDS — so every row lands within a few ms of every other and a latency computed over them would print ~0s everywhere. callCarriesRealTiming treats a whole-call span under 1000 ms as that fallback's signature (no real call fits inside a second) and the whole call reports nothing. A fabricated 0s is the same lie as a fabricated 10.

Text — loop_processing, rendered 2.3s. trace.endedAt − trace.startedAt. What it does NOT measure: the resident's wait. trace.startedAt is stamped by the TraceCollector's constructor, which the agent loop builds after the SQS dequeue and after the escalated-thread gate, so everything before the loop is outside it. The only true receipt stamp is InboundQueueMessage.receivedAt, which is never persisted — so a number claiming to be the resident's wait would be fabricated. The table says "processing time, measured from when the loop started, not from when the message arrived" in words rather than in a glyph. It is endedAt − startedAt and never totalDurationMs, because recordDelivery() refreshes endedAt without recomputing the ms field.

The latency row's grade cell is the en dash. The other three cells in that column are rates; seconds are not a rate, and inventing a pass/fail threshold to make one would put a product judgement nobody has made into a column of measured rates. When some turns have no measurable anchor the row names the shortfall ("9 of 10 turns — the rest have nothing to measure from"); a mean over nine printed as though over ten is the fabricated-number rule at one remove.

9. The closing table posts on every graded conversation

⚠️ SUPERSEDED, 2026-09-18 — grades came off Slack. No closing table is posted at all; the seam posts only the alert card. renderTurnThread still BUILDS the table, for scripts/rerender-scorecard-thread.ts and scripts/render-scorecard.ts. The turnMessages.length >= 1 ownership rule below is retained here as the record of why the condition was narrowed.

The old rule — post the table only when the pass rendered two or more turn lines, or the gate fired — is replaced by turnMessages.length >= 1.

That guard is the idempotency, and it is why the condition is narrowed rather than dropped. turnMessages is derived from trigger.freshRows, and exactly one attempt ever wins a given turn_checks_posthoc row (conditional put), so the table posts on the pass that graded something and never again — the same ownership the turn lines already have, with no new marker and no watermark. A bare !!thread.summary would post a fresh table on every 6-hour trajectory review and every promise resolution, forever, on a conversation nobody has spoken on since.

The || scorecard.gate.fired clause is gone with it. It had no freshness of its own, so a second gate-fired pass with no new turns re-posted the same table — a latent duplicate that existed before this change. A gate-fired pass on a call always has fresh turns anyway.

10. One place, and only one, recomputes a null decision instead of reading it off the row

Everything above insists the null decision comes from the persisted row and fails toward graded when the row is silent. There is exactly one sanctioned exception, withSubjects in scripts/rerender-scorecard-thread.ts, and naming it here is the point of this section — an undocumented exception to a "one definition, read from the record" rule is how the rule quietly stops being true.

The problem it solves: a turn_checks_posthoc row written before this ADR carries no subjects= signal, so turnGradeFromRows answers null and isNullTurn is false on every one of them. That is exactly right for the nightly, whose 7-day window has no ConversationTurn records to consult and must never invent a null. It is exactly wrong for this script, whose entire job is to show a thread graded before the rule as the rule would now render it. Without the fill the run is not a harmless no-op, which would at least be obvious — it is worse: the latency lands on every line, the table gains its latency row, the edits apply and the script reports success, and every greeting keeps its 10. It succeeds at everything except the one thing it exists for.

It is a recompute, not a guess, and it is sound because both inputs are pure. buildConversationTurns is pure over the persisted rows (the same rows yield the same turns forever) and the subject probes are pure functions of the turn, so the values it fills are the values a v3 pass would have written — the same values a fresh pass would persist today. Only the two absent fields are filled. Findings, the judge verdicts and toolsFailed still come from the rows, so no number changes that the record did not already imply, and the safety gate in §1 still holds unchanged: a turn with a deduction on it cannot be nulled by this path either. The injection snapshot is not consulted and does not need to be — a figure is a SUBJECT whether or not anything backed it, and backing decides findings, which this does not touch.

The script prints how many turns it recomputed and why, in the dry run as well as the send, because the person reading the dry run is the person who needs to know these subject sets were not read off the row.

This exception does not generalize. It is available only to a tool that is re-rendering an already-decided thread from rows it also re-reads; nothing that computes a metric may use it, because a metric's whole claim is that it counted what was recorded.

Entity classification

No new persisted entity. subjects / subjectTools are two new signals on the existing turn_checks_posthoc MachineGrade row; latencyMs / latencyKind are derived fields on the in-memory ConversationTurn (rebuilt from Message / AgentTrace rows on every pass, drift tolerance none). NULL_ROOT_STORED = 0 is a sentinel value in the existing Conversation.scorecardGrade field, not a new field.

Consequences

What the fixture now renders (tests/fixtures/voice/pricing-before-lookup.json, the anonymized Camellia call — six turns null, four graded; call grade 3 where it read 5):

turn 1  :grade_null_circle: · –

turn 2  :number_6_circle: · ~3s
said she would check pricing and availability, then only transferred the call

turn 3  :grade_null_circle: · ~6s
...
*Call grade :number_3_circle: — 2 of 4 turns clean, lowest turn scored 2*

| layer | grade | what it saw |
|---|---:|---|
| context | – | doesn't apply on a call — there is no written brief to check |
| tools | :number_10_circle: | 3 tool calls across 10 turns (transferred the call, booked a tour, ended the call) — none failed |
| reply | :number_5_circle: | 1 of 4 turns said she would look something up and did not; 1 of 4 turns quoted a price or special nothing backed |
| latency | – | she answered ~7s after the caller on average, ~19s at the slowest (9 of 10 turns — the rest have nothing to measure from) — measured from when the caller started speaking, so it is what they waited |

null: 6 of 10 turns had nothing to check — a greeting, a question, a closing — and are not in the call grade

The call grade moved from 5 to 3 without a single check changing its verdict. The denominator changed: "8 of 10 turns clean" was two findings against eight passes, six of which were hellos. "2 of 4 turns clean" is the same two findings against the turns that actually had something in them.

The honest negatives:

Slack metadata changes (read by the 👍/👎 path): the per-turn conversation_turn_grade payload's grade is '' on a null turn — never the literal string "null" — and gains nothingToCheck. The conversation_grade_summary payload gains nullTurns, and its grade is '' when every turn was null.

Follow-ups this implies: the four Camellia threads graded 2026-09-07/08 are re-rendered in place with scripts/rerender-scorecard-thread.ts (dry run read first, then --send once BOTH of that thread's seam drivers have closed — the grader workflow AND the promise workflow; the script checks both and refuses otherwise); and scripts/eval/replay-turn-checks.ts is re-run over its 30-day window under v3, so the measured numbers in per-turn-grading.md describe the current population rather than v2's. That page is the canonical description of the mechanism and is updated alongside this ADR.

Alternatives considered