The Oracle — synthesized design (Phase 1 output)
Status: FINAL + Architect ruling 2026-09-08 ~03:50Z (fifth bucket grader gap; plain-English findings). (judge A: smallest > safety = dataflow; judge B: dataflow > smallest = safety — the two verdicts converge on one shape, taken here). Everything lives inside D1–D11; the places the panel refined a decision's wording are in §10 with the reason.
Shape: the smallest-diff skeleton (speaker + promptRule, reuse of every existing seam), the data-flow bundle (pre-joined scalars, replay.agrees, provenance note, the TS script as the ONE decider), and the safety failure discipline (breaker, env allowlist, minted-only, claim_send, draft fallback, _no_oracle_shell floor). Both judges rule the flagged call prompt gap (verified by me — §3).
1. Data flow, end to end
postScorecardIfNew step-2 loop (propflowai, scorecard-post.ts:438-452)
post() now returns notify()'s ts
grade < ORACLE_THRESHOLD (=8, exported from turn-thread.ts beside REASON_LINE_BELOW)
startedAt within 72h → deps.onLowTurn(payload) fail-soft, 2s race, never throws, outcome logged
│ default impl = src/lib/temporal/activities/oracle-door.ts (SMITH_TEMPORAL_* env, own Connection,
│ circuit breaker 5 min, closed-union outcome: started|already_started|refused:<reason>)
▼
Temporal tools-prod OracleTurnWorkflow id=oracle-<conversationId>-<turnKey> queue propflow-smith FAIL
A1 smith_oracle_evidence → shells `npx tsx scripts/oracle-evidence.ts --conversation <id> --turn <key> --json`
from the pinned worktree (verify-wt.sh shape), env ALLOWLIST (no Slack/Temporal/Anthropic/GH tokens,
no ELEVENLABS key), heartbeat 10s, scrub + schema + PII guard on the way back
pure render(bundle) + validate(text) (workflow sandbox; the BUCKET is read from the bundle — the TS script is the one decider, Python never re-decides)
A2 smith_oracle_claim (PR-bearing buckets only): gh head-search → try_claim (act on `minted` only)
A3 smith_slack_send(thread_ts=rootTs, source="oracle-turn") retry 3 — claim_send (outbound_ledger.py:687) makes retries at-most-once
child SmithReplyWorkflow (ABANDON, 90 min) with build_oracle_fix_trigger(...) only if PR-bearing ∧ minted ∧ no PR
chat_jid = SLACK_AGENT_SMITH_CHANNEL, NEVER the property thread (Smith's own ack/reply there would count as a second finding)
link wait: workflow.sleep(10 min) × ≤9 → head-search → smith_slack_send(update_ts=findingTs, "Fix: <url>")
A4 smith_oracle_log → ~/.claude/smith-state/oracle/dispatch.log (SYSTEMS.md signal)
automation_run("oracle-turn", kind="detector") ok for posted|refused:*, fail for failed:<stage>
The nightly: NightJobSpec(job_id="oracle-nightly") in the MORNING queue under workflow.patched("oracle-nightly-v1"), placed immediately before smith-review (which pins itself last), running python -m agent_smith.oracle_cli nightly → shells npx tsx scripts/oracle-nightly.ts --json → one slack.post_message line per property under the banner (SMITH_QUEUE_THREAD_TS) → oracle-nightly posted log line → automation_run("oracle-nightly", kind="cron").
The flagged call (D10): python -m agent_smith.oracle_cli run --conversation <id> --turn <key> [--dry-run] builds the same payload from prod rows (Conversation slackTs/slackChannel, the checks row's gradedAt; turnLineTs found by one conversations.replies scan for the turn N line, falling back to the root) and starts the SAME workflow id with id_conflict_policy=FAIL; --dry-run prints the id + payload and never builds a client.
2. The hook payload (D5) — closed set, primitives only
{conversationId, propertyId, turnKey, turnOrdinal, grade, gate, channel, rootTs, turnLineTs, startedAt, gradedAt, rubricVersion, postedAt, source:'seam'|'cli'}. Never: m.text, reasons, inbound, response, any tool step, any judge detail, names, phones. Pinned by oracle-payload.drift.test.ts (key set + primitive types).
The 72h floor (R-7) is evaluated in the seam before the hook is called (refused:backfill_floor, no Temporal call), so a backfill storm never reaches tools-prod.
3. The decision table (Q2) — inputs are bundle fields only
Inputs: findings[] (from the persisted row via turnGradeFromRows), followedByClass ∈ {transfer_agent, transfer_number, lookup, book, record, none}, speaker {slug|null, via: 'transfer_result'|'position'|null}, boundForSpeaker[], fulfillingForSpeaker[], fulfillingElsewhere[], backing[], injection.source, subjectBacked, promptRule ∈ {scripts, forbids, absent} (over the SPEAKER's prompt file at HEAD, per check key; for intent_unfulfilled + transfer_agent, scripts = the subject sentence verbatim (case/punctuation-insensitive) OR the file's hand-off rule requiring a first-person "let me check…" bridge — so a paraphrased hand-off never reads as a missing tool on the tool-less router), toolsFailed, policyGate, fairHousing, rowPresent, gradeRecomputed, replay.agrees, and — for every followedBy = transfer_agent finding — handoffDelivered ∈ {true, false, unknown} = whether the RECEIVING agent's own/backing tool calls, or its injection-backed figures, carried the promised subject (the LOOKUP_HEADS tools regex for the subject's heads; backed_by=injection on a later turn's row counts) in ANY later turn of the same call; unknown when the receiving speaker cannot be resolved or the call has no later turns. The bundle also carries handoffDeliveredAtTurn (ordinal) and handoffReceiver (slug).
Buckets (five + the honest outcome): prompt gap (layer prompt), missing tool (layer tool binding / tool handler / personalization injection), tool result ignored (no PR), policy (no PR), grader gap (layer detector — the check fired on a shape the grader does not follow; PR allowed and held, scope fleet, target the detector file), could not determine (no PR).
Speaker derivation (the input no designer had verified; I did): voice transfer_to_agent tool_result rows carry from_agent/to_agent (prod and fixture). A turn AFTER a transfer result belongs to that result's to_agent (mapped via AGENT_ID_TO_SLUG, agents/clara/lib/agent/specialists/registry.ts:134, or the property's clone fleet in fleets.ts); a turn BEFORE the call's first transfer_to_agent follow-up is the entry agent (triage) by position; an unmapped id after a transfer → speaker = null (row 11). Text turns: the trace's agent (leasing/maintenance by workflow).
Precedence: 1 → 2 → 3/4/5 → the highest-weight finding's row; a tie across buckets → row 21.
| # | Evidence condition | Bucket | Fix layer | PR | Settles "could not determine" |
|---|---|---|---|---|---|
| 1 | no turn_checks_posthoc row (R-10) |
could not determine | none | no | a graded row |
| 2 | gradeRecomputed ≥ 8 (R-8); text leads with both grades + rubric |
could not determine | none | no | — |
| 3 | fairHousing = red (gate) |
policy | none | no | — (judge detail NEVER quoted) |
| 4 | policyGate = red, no Clara finding |
policy | none | no | — |
| 5 | grade 7: no finding, toolsFailed and/or policyGate = yellow; text says "no reason line exists at 7", names the failed tool NAME |
policy | none | no | — |
| 6a | intent_unfulfilled, followedBy transfer_agent, speaker known, promptRule = scripts, handoffDelivered = true — the receiving agent delivered the subject at a later turn |
grader gap | detector (src/lib/domain/conversations/turn-checks/stated-intent.ts / adapter.ts — treat a sibling hand-off whose subject is delivered later as continuation) |
yes, held | — |
| 6b | same, handoffDelivered = false — flagged call, fixture #t2 (the call's later turns carry no pricing lookup or injection-backed figure: prod #t7 is backed_by=injection → check whether that flips 6b→6a at implementation time and record which) |
prompt gap | prompt (speaker's file) | yes, held | — |
| 6c | same, handoffDelivered = unknown |
could not determine | none | no | the receiving agent's later turns |
| 7 | intent_unfulfilled lookup class, bound = true, no fulfilling call, promptRule = absent |
prompt gap | prompt | yes | — |
| 8 | as 7 but promptRule = forbids (a leasing-spoken "let me check… then transferred" lands here; text quotes the existing rule) |
could not determine | none | no | served prompt == HEAD for the file AND an eval replay reproducing it → then a strengthening PR |
| 9 | intent_unfulfilled lookup class, bound = false, promptRule ≠ scripts, a VOICE_TOOL_CATALOG spec exists unbound for this agent |
missing tool | tool binding | yes | — |
| 10 | as 9, no spec at all | missing tool | tool handler | yes | — |
| 11 | intent_unfulfilled, subject has no head OR speaker = null OR (followedBy transfer_agent on a router agent whose prompt has no hand-off rule) |
could not determine | none | no | a head word / a persisted agent id per turn / a hand-off rule — the router is tool-less by design (specialist-tool-coverage.test.ts:83-92), never a binding PR |
| 12 | intent_unfulfilled book/record/transfer class: FULFILS tool bound & not called → prompt gap (promptRule ≠ forbids, else row 8); unbound → missing tool |
prompt gap / missing tool | prompt / binding | yes | — |
| 13 | numeric_mismatch |
tool result ignored | none | no | — |
| 14 | unbacked_figure/unbacked_time/unbacked_action_claim with a successful non-control call in the relevant window that does not carry the subject |
tool result ignored | none | no | — |
| 15 | unbacked_figure/unbacked_time, no backing call, injection.source = trace, subject ∉ known figures/slots: bound = true → prompt gap (or row 8); bound = false → missing tool / personalization injection |
prompt gap / missing tool | prompt / injection | yes | — |
| 16 | unbacked_figure/unbacked_time, no backing call, injection.source ∈ {elevenlabs, none, prompt} — fixture #t7 |
could not determine | none | no | a persisted AgentTrace.injectedContext on the call |
| 17 | unbacked_action_claim: a window tool failed → tool result ignored; nothing ran, bound → prompt gap; unbound → missing tool |
see row | see row | see row | — |
| 18 | tool_error_as_policy (tool NAME + error class only) |
tool result ignored | none | no | — |
| 19 | internal_narration (text): promptRule = absent → prompt gap on the text prompt; forbids → row 8 |
prompt gap / CND | prompt / none | yes / no | as row 8 |
| 20 | replay.agrees = false (row vs re-run finding sets differ) |
could not determine | none | no | the same rubric on both sides |
| 21 | top-weight findings tie across different buckets | could not determine | none | no | — (lists all, no mechanism) |
policy, tool result ignored and could not determine open no PR (D7). PR-bearing rows: 6a, 6b, 7, 9, 10, 12, 15, 17, 19. Both 6a and 6b PR bodies cite triage.ts:457 (rule 5 scripts the line), triage.ts:482 (owner rule: first person, never announce a transfer) and leasing.ts:236 (the sibling is expected to deliver pricing), name both candidate layers, and say the human picks; the proposed diff is the row's layer, held.
Flagged call, decided: intent_unfulfilled · subject "check pricing and availability" · followedBy=[transfer_to_agent] (control tool) · own/backing empty · speaker = triage (position: turn index 1 precedes the first transfer_to_agent; prod from_agent = triage AGENT_ID corroborates) · boundForSpeaker ∩ fulfilling = ∅ (triage floor is identify_caller, escalate_to_human) · fulfillingElsewhere = get_available_units, get_property_details, get_term_pricing on leasing · promptRule = scripts (triage.ts:457 rule 5 prescribes the exact sentence; :480-491 owner rulings) → row 6: prompt gap / prompt / agents/clara/lib/voice-agents/triage.ts / PR yes, held. Why not "could not determine": nothing about the mechanism is unknown — the prompt prescribes the sentence, the speaker cannot keep it, and the detector's window (followUpSteps, tool calls only) can never see the sibling keep it. Whether the FIX is desirable is what hold-for-review puts in front of a human. Provenance (judge A, verified): nearest merge before the call 43f5574bd9; triage.ts blob 2d68c9a2ff there vs 4a3ba37bc3 at HEAD (#7303, merged after the call); the rule line is byte-identical in both; leasing.ts unchanged. The finding says so.
4. Q5 — cause signature
sig = sha256("oracle-v1|" + checkKey + "|" + followedByClass + "|" + speaker + "|" + fixLayer + "|" + fixTarget).hexdigest()[:16]
fixTarget= repo-relative prompt file (prompt) / the tool NAME (tool binding) / handler file (tool handler) /personalization(injection) / the detector file (detector, e.g.src/lib/domain/conversations/turn-checks/stated-intent.ts) /-(none).fixLayer ∈ {prompt, tool binding, tool handler, personalization injection, detector, none}.- No subject phrase, no figure value, no property, no utterance, no commit sha, no bucket. "check pricing" and "check pricing and availability" before a transfer are ONE cause (one triage rule) → one PR; the same detector on a different file is a different cause. Over-merging costs one PR body listing two related turns (later fix-drives resume the PR); under-merging costs a duplicate fleet PR — the cheaper error is chosen.
- Flagged call pre-image:
oracle-v1|intent_unfulfilled|transfer_agent|triage|prompt|agents/clara/lib/voice-agents/triage.ts; fixture#t2yields the same string (the D6 positive control). - Claim key
oracle:<scope>:<bucket-slug>:<sig>viawork_claims.try_claim(work_claims.py:478);scope = fleetfor prompt / tool binding / tool handler / personalization-route targets, the ticker only for per-property DATA (none in the table today). Branchsmith-fix/oracle-<scope>-<bucket-slug>-<sig>; head-searchgh pr list --repo PropFlow-Technologies/propflowai --state open --search "head:smith-fix/oracle-<scope>-<bucket-slug>-<sig>". Race: A wins the claim and starts the child; B minutes later finds no PR,try_claimreturnswon=False→ no child; B's finding says the cause is already being driven and the link wait attaches the URL when it exists. Awon=True, minted=False(RPC error,work_claims.py:544-549) NEVER starts a child. - The sig and the branch are never rendered in the finding (a 16-hex could be all digits and match the thread instrument's phone regex); only a PR URL is.
5. Q6 — property scope for prompt PRs
Allowed, fleet-scoped, disclosed, held — never re-asked of the Oracle. A property not in AGENT_FLEETS_BY_PROPERTY uses the single shared registry (fleets.ts:46-58), so a per-property prompt PR would be a fiction. The claim is fleet-keyed; the branch slot is fleet (or the ticker for per-property data), never the 13-digit id. Three gates on the PR: (1) gh pr create --label hold-for-review then gh pr merge --disable-auto, both read back with gh pr view --json labels,autoMergeRequest,isDraft and asserted; on any assertion failure gh pr ready --undo (draft — GitHub itself refuses auto-merge); (2) the body's first section names the blast radius in the reader's words ("merging changes what every shared-fleet property hears; merge is the deploy") and the layer; (3) a single-turn eval that fails on the pre-change prompt is mandatory (never no-eval-needed:). The Oracle never merges.
6. Q9 — the nightly denominator
- Windows are instants:
W0 = [run−24h, run),W7 = [run−8d, run−24h),run= the step's start. - ONE read:
getMachineGrades(undefined, {since: iso(run−8d)})— the property-lesssincepath is the bounded GSI3 window across every partition, UNASSIGNED included (grading.ts:441-457; with a propertyId the same call is a full append-only partition scan filtered in memory, judge B §9). Group client-side bypropertyId, then bytrace:key. Properties =getProperties()with!isTest; every live property gets a line. Denominator = turns MEASURED in the window: keys whoseturn_checks_posthoc.gradedAt ∈ W0; grade =computeTurnGrade(turnGradeFromRows(rowsOfKey, key)).grade.avgone decimal,below8 = |grade < 8|,avg7pooled over W7,Δ = avg − avg7signed one decimal,—when either n = 0. A backfilled turn counts on the day it was graded — "measured", not "spoken" (startedAtis not on the row; deriving it means a conversation read per key — accepted cost, see unmeasured). - Unmeasured (primary): for each conversation with ≥1 row in W0,
buildConversationTurns(getConversation(id), getAgentTracesByConversationId(id)).turnsminus keys with a checks row — the only count that sees a turn the judges never touched; a bounded number of reads per night. Fallback when a read fails: rows-only (keys with a judge/internals row but no checks row), marked as such. Conversations with NO row at all are invisible to both — the line's footer says so. UNASSIGNEDis not a property: one trailingunassigned: N turnsline when N > 0.isTestproperties excluded.- Line (contains the instrument's four literals
avg,below 8,7d,unmeasured):oracle nightly · CAM · avg 8.4 over 17 turns measured in the last 24h · below 8: 3 · 7d avg 8.1 (Δ +0.3) · unmeasured: 2 (calls with no grade rows are not counted)n = 0:oracle nightly · CAM · no turns measured in the last 24h · 7d avg 8.1 · unmeasured: 0. Never a fabricated 10.
7. Q10 — queue, activities, timeouts, returns, logs
OracleTurnWorkflow on propflow-smith (SMITH_WORKFLOWS; brand-new class, no patch gate), execution_timeout = 120 min (covers the link wait; the finding itself posts within D1's 15 min before any wait).
| activity | shape | timeouts | retry | idempotent |
|---|---|---|---|---|
smith_oracle_evidence(payload) |
sync def; git fetch origin main && checkout --detach origin/main ≤45s (keep HEAD on failure, flag worktreeStale); npx tsx scripts/oracle-evidence.ts … --json ≤240s; reader thread + heartbeat 10s; env ALLOWLIST (PATH HOME AWS_* DYNAMODB_TABLE_NAME=propflow-prod DATA_BACKEND=dynamodb); parse last stdout line; schema + bucket enum + quotes[i] ∈ json.dumps(evidence) + forbidden-key walk + ≤32KB + scrub_phones/scrub_secrets |
s2c 6 min, hb 60s | 3, backoff 10s | yes (read-only) |
pure render / validate |
workflow sandbox; bucket/fixLayer/pr/causeSig are READ from the bundle (one decider); validate = the ONLY " characters are the ≥12-char spans, each ∈ quotes and ⊂ json.dumps(evidence); no \+?\d[\d\-\(\) ]{8,}\d; no @, &, <, >; no markdown control chars except single-star bold on the verdict line; no ISO date; ASCII; shas 7 chars; ≤12 lines; bucket literal once; failure → refused:finding_invalid (loud in the AutomationRun, nothing posted) |
— | — | yes |
smith_oracle_claim(workKey, branch) |
async; gh pr list head-search then try_claim via worker.build_client (floor-covered); returns {existingPr, won, minted} |
s2c 60s | 2 | claim re-entrant per owner |
smith_slack_send (existing) |
finding post thread_ts=rootTs, source="oracle-turn"; later edit update_ts=findingTs |
s2c 60s | 3 — claim_send (outbound_ledger.py:687) keys the fresh post on (workflow, run, activity) so a retry can never double-post |
at-most-once |
child SmithReplyWorkflow |
by string name, task_queue=propflow-smith, ABANDON, 90 min, attempts 1; trigger from pure build_oracle_fix_trigger(bundle, findingTs, workKey) with chat_jid = SLACK_AGENT_SMITH_CHANNEL (the build_remediation_trigger shape, remediation_prompt.py:190-205) — never the property thread; the prompt FORBIDS re-diagnosis and carries the D7 brake + read-back + draft fallback + blast-radius paragraph + eval requirement |
— | — | head-search + claim |
smith_oracle_log(line) |
append YYYY-MM-DD HH:MM:SS oracle-turn <outcome> conv=<id> turn=<key> bucket=<b> to SMITH_STATE_DIR/oracle/dispatch.log |
s2c 10s | 1 | yes |
Returns OracleTurnResult{outcome: posted|posted_linked_existing|refused:<reason>|failed:<stage>, bucket, causeSig, findingTs, prUrl, evidenceMs} — ids and enums only. Signals (SYSTEMS.md oracle, proposed, log-match-age only): oracle-turn (staleAfterS 14d — low turns are sparse) and oracle-nightly posted (staleAfterS 30h — the true liveness). automations.toml: oracle-turn (kind="detector", category="conversation", runtime="temporal", namespace="tools-prod", workflow="OracleTurnWorkflow", channels=[]) and oracle-nightly (kind="cron", workflow="MorningQueueWorkflow", trigger="morning-queue step 4.75 — daily 07:00 America/Chicago", channels=["agent-smith"]).
8. The finding writer: deterministic template (unanimous)
No LLM in the finding path. Every "…" span is an element of evidence.quotes[] and those are the ONLY " characters in the text (a stray quote shifts the instrument's pairing); no &<> (Slack escapes them in text), no markdown control characters — single-star bold is allowed on the verdict line only; nothing else in the finding carries mrkdwn — shas at 7 chars, no sig/branch/ISO date/ts/13-digit id anywhere, turn N (#tN) literal, bucket literal once. The LLM lives only inside the fix-drive brain, which receives bucket + target + excerpt and is told not to re-diagnose. Flagged-call text (straight quotes; no run of 10+ digits; no ISO dates):
Oracle · turn 2 (#t2) · graded 6 when the line was posted; the rows still read 6 today · Cause: prompt gap
What she said: "Let me check pricing and availability for you." — then handed the call to the leasing line; nothing was looked up on this turn or before it.
What the grader flagged: "said she would check pricing and availability, then only transferred the call"
After the hand-off: the leasing line did not look up or quote pricing on any later turn of this call (so the promise was not kept across the hand-off). [6a variant: the leasing line quoted pricing at turn 4, so the promise WAS kept across the hand-off — the check fired because it does not follow hand-offs. Cause: grader gap.]
Caller side: asked about pricing, availability (topics from our vocabulary table; the caller's words are never quoted).
Tool list checked (voice catalog at HEAD 83d9fba): the front-door agent (triage) binds no pricing or availability tool; leasing binds get_available_units, get_property_details, get_term_pricing.
Prompt checked: agents/clara/lib/voice-agents/triage.ts read at HEAD 83d9fba; this file changed after the call (nearest merge at call time 43f5574); the quoted rule is present in both. Rule 5 scripts this exact sentence as the hand-off line: "Let me check pricing and availability for you"
Briefing: not persisted on this call (pre-field); read back from ElevenLabs at grade time — it does not bear on this finding.
Where the fix goes: the front-door prompt's hand-off wording (or, if the promise was kept later: the grader, so it follows a hand-off). Either way it is a shared-fleet change — a merge deploys to every property — and the wording is an owner ruling, so the PR is held for a person to decide.
Fix: opening a held PR for this cause — or: Fix: PR already open for this cause: <url> — or, after the wait: Fix: <url>
The block above is the design-time sketch, not the shipped text: the renderer that shipped in agent-smith #437 leads with the verdict in plain words and trails one Receipts: line, and it is ASCII (-, never · or —). Plain English throughout (Architect): no detector keys, no field names, no "bucket" — "Cause: prompt gap" style. Variants: R-8 leads with "graded N when posted; now M under could not determine swaps lines 5–9 for "Could not determine: tool result ignored says the tool NAME returned a result that does not carry the subject, no PR; policy names the judge level, never its detail. The caller line falls back to "Caller side: no topic word matched (paraphrase withheld)."
9. scripts/oracle-evidence.ts --json (D3) — key order is the contract, built literally
{ schema:"oracle-evidence-v1", mode:"prod"|"fixture", conversationId, propertyId, channel,
turn:<1-based ordinal as the thread line renders it>, turnKey, startedAt,
grade:<number, recomputed>, gradeDetail:{ rowGrade|null, rubricVersion, rowRubricVersion|null, gate, deductions, reasons[] },
bucket, fixLayer, fixTarget|null, pr:boolean, undetermined:<what would settle it>|null, causeSig,
quotes:[<every scalar the template may quote — ASCII, no " \ newline, each asserted ⊂ JSON.stringify(evidence)>],
evidence:{
responseQuote|null, responseQuoteWithheld:""|"non-ascii"|"capitalized token"|"no firing sentence",
inboundTopicsText, inboundPresent,
findings:[{key, subject, reason, followedByText}], checked[], checkedText,
speaker:{slug|null, via:"transfer_result"|"position"|null},
tools:{ ownText, followUpText, backingText, ownFailedText }, (names + outcome only; never toolInput/toolResult)
internals:{tools, context, loop}|null, policyGate|null, fairHousing|null,
injection:{source, knownFigureCount, knownSpecialCount, tourSlotsPresent, backedByInjectionText},
replay:{findingsText, agrees},
catalog:{ boundForSpeakerText, fulfillingForSpeakerText, fulfillingElsewhereText, unboundCandidatesText },
prompt:{ file, rule:"scripts"|"forbids"|"absent", excerpt, line, blobAtCall, blobAtHead, changedSinceCall }|null,
anchor:{channel, rootTs}|null },
catalogAsOf:<HEAD sha>, promptAsOf:<HEAD sha>, nearestCommitAtCall:<sha>|null,
provenanceNote:"read at HEAD; <file> unchanged since the call"|"read at HEAD; <file> changed after the call; quoted rule present in both"|"read at HEAD" }
Rules: findings come from the ROW (turnGradeFromRows); replay re-runs runTurnChecks(turn, nextTurnOfCall(all, turn)) with no options plus a second run with snapshotFromInjectedContext iff the trace carries injectedContext — never the ElevenLabs live read (non-deterministic; today's value, not the call's). --turn accepts the ordinal (N-th Clara turn in conversation order) or a key (contains #t or starts with trace_). Prod reads via @/lib/data only (never store.ts), explicit DYNAMODB_TABLE_NAME, and the first stderr line names the table. responseQuote = the firing sentence only (the sentence where detectStatedIntent/foldSpokenNumbers reproduces the subject), then redact() + maskKnownNames(knownNameTokens(messages) ∪ name tokens off the Conversation/Person row), then withheld (null, reason recorded) if any non-ASCII, ", \, or a capitalized token other than sentence-initial / I / Clara / the property-name tokens survives. excerpt ≤160 chars, extended to the nearest "/backtick/newline. **Never in the bundle**: toolInput, toolResult, inbound, contextBefore, response in full, judge detail, transfer condition, Conversation beyond id/propertyId/channel — enforced by the output type AND a JSON-walk test. Git reads (rev-list -1 --before=<startedAt> origin/main, rev-parse HEAD, rev-parse <c>:<file>) are injected deps so the unit test stubs them; both instrument runs share one worktree, so output is byte-identical. No Date.now(), no env-dependent ordering, no unordered sets.
10. Where the panel refined D1–D11 wording (not re-opened)
- D6/D7 branch
<property>slot =fleetfor shared-fleet targets (or the ticker for per-property data), never the 13-digit id — the dedupe must be fleet-wide for a fleet-wide change, and a 13-digit run matches the thread instrument's phone regex. - D8 "appended" = appended to the morning queue, inserted immediately BEFORE
smith-review(whose own contract and pinned test say it is last). - D2 "the LLM only WRITES" → no LLM at all in the finding path; a template over the bundle has no fabrication path. The LLM stays inside the fix-drive brain.
- D11 sequencing: the code reads
SMITH_TEMPORAL_*fail-soft from day one; thetask-definition.template.jsonedit is a separate PR that lands only AFTER the secret exists (preflight-secrets.pyfails every worker deploy on an unreadable secret). - Brief §3.2 correction: fixture
#t2isfollowed_by=transfer_to_agent(nottransfer_to_number); fixture and flagged call are the same decision row — the D6 positive control. - Brief §3.1/D8 correction:
getMachineGrades(propertyId, {since})is a partition scan filtered in memory; the GSI3 window is the property-lesssincepath — the nightly reads it once and groups client-side. - D2 "the decision table" lives in ONE place:
scripts/oracle-evidence.tsdecides; Smith renders. A Python re-derivation would be a second decider whose drift the flagged-thread instrument (which diffs the posted bucket against an independent TS run) would fail.
11. Tests (name → what fails without the code)
propflowai: scorecard-post.test.ts (+): hook fires once per fresh turn < 8, AFTER the line's notify resolved, turnLineTs = that mock's ts; not for grade 8; not for isTest/test traffic; 73h-old startedAt → refused:backfill_floor and the hook is never called; a throwing / 3s-hanging hook leaves turnsPosted, the summary and the outcome byte-identical (positive control: remove the race → timeout). oracle-threshold.drift.test.ts: ORACLE_THRESHOLD === 8 exported from turn-thread and imported by scorecard-post. oracle-payload.drift.test.ts: exact key set, primitives only. oracle-door.test.ts: env absent → refused:no_tools_prod_client without dialing; dial rejection → outcome + breaker closed (second call refused:door_closed, no dial); AlreadyStarted → already_started; the door never imports getTemporalConnection. seam-hook predicate as vitest. oracle-evidence.test.ts: fixture #t2 → prompt gap, speaker.via = 'position', every quote ⊂ JSON.stringify(evidence)+fixture, two runs byte-equal, lowercase JSON lacks alex|rivera|555-|5550100; positive controls: promptRule stubbed forbids → row 8; drop the transfer step → row changes; a response with a caller-row capitalized token → withheld; deleting maskKnownNames → fails; #t7 → row 16 with undetermined naming injectedContext. oracle-evidence-pure-imports.test.ts: the script imports nothing under @/lib/platform/observability or @/lib/temporal. oracle-nightly.test.ts: pure nightlyLines(rowsByProperty, properties, T0) over rows from the REAL builders: arithmetic, isTest absent, UNASSIGNED trailing only, zero-turn never reads 10, unmeasured = 1 for a 3-turn conversation with 2 checks rows.
agent-smith: test_oracle_decide.py (one case per table row; each PR-bearing row flips to CND when its settling field is nulled); test_oracle_finding.py (the §8 text from the fixture bundle; validate REJECTS a 13-digit id, a ts, +1 (206) 555-1234, an email, a span not in quotes, a 13th line); test_oracle_workflow.py on wf_env (posts exactly one finding; same id twice raises; CND → no claim, no child; existing PR → link, no child; won, minted=False → no child; link wait edits with update_ts, never a second fresh post); test_oracle_env.py (the child env key set equals the allowlist); conftest.py (+) autouse _no_oracle_shell (the ONE subprocess seam evidence/git/gh share raises); test_worker_registration.py (+) pins OracleTurnWorkflow interactive; test_morning_queue_workflow.py (+) records (ORACLE_NIGHTLY_JOB.job_id, _BANNER_TS), _ORDER[-1] still smith-review, replay of a pre-change history green with patch_ids lacking oracle-nightly-v1; test_oracle_cli.py (--dry-run never touches build_client; nightly line carries avg, below 8, 7d, unmeasured).
Rung-2 real-path proof: the D10 CLI run on the flagged call IS the E2E; the seven instruments are the receipt.
12. PR plan (Phase 2 — unchanged from the brief, one PR per repo, in order)
(a) propflowai: seam hook + ORACLE_THRESHOLD + oracle-door.ts + scripts/oracle-evidence.ts + scripts/oracle-nightly.ts + config/automations.toml rows + docs/architecture/the-oracle.md + ADR-0132 + tests. (b) agent-smith: oracle_turn.py (pure), workflows/oracle_turn.py, activities/oracle_turn.py, oracle_cli.py, ORACLE_NIGHTLY_JOB + morning-queue step under a patch gate, registrations, tests. (c) agentflow: SYSTEMS.md oracle entry (proposed) + ADR-0014. (d) later, after D11 provisioning: the one-line task-definition template edit.