ADR-0063: Restore the turnover voice call to capture-only (regression post-mortem + harden)

• Status: Proposed
• Date: 2026-06-18
• Deciders: Fede (owner), Jose (Maestro), eng
• Supersedes / relates: 0034 (turnover orchestration), 0044 (close-out), 0048 (branch-test live voice agents), 0053 (Clara dumb-IO / Maestro), 0057 (voice promises require real delivery), 0057 (PropFlow owns dispatch), 0058 (tools must be real or not tools), 0059 (vendor dispatch), 0061 (Maestro package→reasoning→action-queue).

This is a regression post-mortem, not a redesign. The architecture below — capture-only live call → post-call Maestro orchestration → PM-confirm-gated dispatch — already existed and worked. It regressed over 2026-06-16..19 when action tools were wired onto the live call. The owner's words: "before a few days ago it was working great, we regressed." The only genuinely new element proposed here is the Haiku pre-check (Jose's Maestro, ADR-0061, accepted 2026-06-18).

1. TL;DR

1. The live turnover call must do exactly one thing: capture notes. No charges, no dispatch, no actions of any kind until after the caller hangs up. Principle (owner): "least amount of tools possible for the live call; all heavy lifting is after the call with Maestro."
2. The live agent regressed because action tools (dispatch_work_order, add_charge, apply_charge_decision, get_turnover_status, confirm_turnover, send_summary) were added to its in-call tool belt — so the LLM reaches for them mid-walk instead of capturing.
3. The regression shipped green because the test harness mocked every tool and asserted nothing about which tools the agent chose to call. The real-call ElevenLabs E2E harness the owner asked for 10+ times — replay every real call through the live agent as if the PM were calling — was never built. That gap, not the individual bugs, is the headline.
4. Fix = restore + harden (this ADR proposes; does not yet apply the live-agent change):
   • P0 — strip the live agent back to 6 capture-only tools, through the sync pipeline (the sync is additive-only and cannot remove tools today — that must be fixed in the same change), branch-tested per ADR-0048.
   • P1 — build the real-call capture-only gauntlet (all real calls → simulated callers → live agent → assert capture-only + clean hang-up) and make it the merge gate for any walk-agent change.
   • P2 — converge the post-call path onto the current Maestro (ADR-0061): hang-up → Haiku pre-check → reasoning → action queue → dispatch/charge, PM "yes" as the confirm gate.

2. What regressed, and when

The agent was born capture-only (PR #1466, 2026-05-26: "ambient-scribe / medical-scribe lineage — capture-only invariant, no $ amounts"). Action capability then crept onto the live voice surface:

By the incident, the live agent (agent_9601…) carried 12 tools — 6 capture/system + 6 action — verified live via the ElevenLabs API. #2434 had deliberately kept charges SMS-only; #2436 is the change that opened the voice surface. (The other two action/recap tools on the belt — send_summary, a V1 tool long since demoted to a no-op, and confirm_turnover, added to voice by #2203 — likewise don't belong on a capture-only call and are removed in P0.)

3. The live incident (verified)

Owner's call conv_1201kveyz963fcysbk7cfx22pf6t (2026-06-19, 124s). Dictation: "charge $150 cleaning; handyman fix a loose toilet; Miracle Method clean the tub, charge $250; handyman patch walls + repaint; that's it." The agent:

• called start_inspection ✓ then went straight to add_charge ×3 — every one failed (INVALID_INPUT: amount expected number, received string), zero charges posted;
• fired dispatch_work_order ×3 mid-dictation, creating work orders the PM never confirmed;
• never called append_notes (captured nothing) and never called end_call (no recap SMS) — the PM hung up;
• told the PM "I'll text you the rundown" as if it had worked — the exact failure ADR-0057/0058 exist to prevent.

4. Root cause

Layer 1 — the live call became an actor. An LLM with a tool will use it. With dispatch/charge tools in the belt and no structural prevention, the model acts the instant the PM mentions work, and skips capture. ADR-0053 already says this is structural, not a prompt-tuning problem: "a removed tool can't fire — a prompt rule is only a hope."

Layer 2 (the headline) — the harness measured the layer that was already fine. Two harnesses existed; neither could catch this:

• CRUCIBLE reliability baseline (evals/turnover-walk/sim-runner/mock-driver.ts) — the PR gate — calls simulate-conversation with tool_mock_config mocking every tool (add_charge → fake {ok:true}) and asserts nothing about which tools were called. Mock-everything + assert-nothing cannot fail on "dispatched mid-call."
• simulate:walk (scripts/simulate-walk-call.ts) fires real webhooks but ships only 6 synthetic scenarios and asserts only notes integrity — no capture-only assertion.
• The 41 real harvested calls (evals/turnover-walk/corpus/) were only ever replayed offline — never sent back through the live agent.
• The leasing side already has the right pattern (scripts/voice-harness/drivers/l1-simulate.ts, test:voice-leasing); turnover never got the equivalent.

The owner asked 10+ times for exactly this gate — "use all the test calls and real ones from Camellia and run them through a simulator as if it was me calling… remove myself from the test loop." It was never built.

4.1 Proof — the capture-only gauntlet (run 2026-06-19)

We replayed all 41 harvested real calls back through the current live agent as simulated callers (tool returns mocked = zero prod writes; tool choices asserted). Result: 5 hard capture-only violations — on real PM dictation the live agent fired action tools mid-walk:

Of the 41 calls: 16 PASS, 5 capture-only violations, 20 mock-fidelity FAILs. The 20 were artifacts — generic {ok:true} mock returns broke the agent's unit-resolution so the sim looped before capturing; not agent bugs. PR #2499's committed harness fixes this with faithful resolution mocks (realistic find_turnover/start_inspection returns) while still mocking the write tools and asserting tool choice — validated: the 3 clean calls PASS and all 5 violators FAIL. The lesson holds: a harness that asserts on what the agent chose to call catches this deterministically; the prior harness only checked that capture happened, never that actions didn't.

Why structural drift detection didn't catch it: the nightly el-config-drift-live check ran the morning of 2026-06-19 and turnover_intake PASSED — every catalog tool bound, schema/url/description matching live. The regression is invisible to drift detection because the action tools were legitimately added to the catalog (#2434/#2436); a catalog⇄live parity check has no opinion on whether a capture-only agent should carry action tools. Parity ≠ appropriateness. (The drift alert that did fire that night was unrelated — schema/description drift + missing SMS tools on the renewal / leasing / lease-billing / unknown-caller agents.)

5. Current (regressed) architecture

flowchart TD
    PM([PM places turnover walk call]) --> Triage[Clara — Triage]
    Triage -->|transfer_to_agent| Walk["Clara — Turnover Intake (agent_9601)\n12 TOOLS"]

    subgraph LIVE["LIVE CALL — acting mid-walk (REGRESSED)"]
      Walk --> SI[start_inspection]
      SI --> AC["add_charge ×3\n❌ INVALID_INPUT: amount is a string\n→ 0 charges posted"]
      SI --> DWO["dispatch_work_order ×3\n⚠ WOs created mid-dictation,\nnever confirmed by PM"]
      Walk -. "rarely / never" .-> AN[append_notes]
      Walk -. "never" .-> EC[end_call]
    end

    AC --> Hangup([PM hangs up — no recap, nothing captured])
    DWO --> Hangup
    EC -. skipped .-> Hangup

    Hangup -.->|BYPASSED| Post["Post-call recap → PM 'YES' → dispatch\n(the authorized path — never reached)"]

    classDef bad fill:#5a1f1f,stroke:#e05757,color:#fff;
    classDef warn fill:#5a4a1f,stroke:#e0b357,color:#fff;
    classDef skip fill:#333,stroke:#888,color:#aaa,stroke-dasharray:4 3;
    class AC,Hangup bad;
    class DWO warn;
    class AN,EC,Post skip;

6. Proposed (restored + hardened) architecture

flowchart TD
    PM([PM places turnover walk call]) --> Triage[Clara — Triage]
    Triage -->|transfer_to_agent| Walk["Clara — Turnover Intake\nCAPTURE-ONLY · 6 tools"]

    subgraph LIVE["LIVE CALL — capture notes ONLY · no actions until hang-up"]
      Walk --> FT[find_turnover]
      FT --> SI[start_inspection]
      SI --> AN["append_notes\n(findings + PM-stated amounts + vendor names\ncaptured AS NOTES, not actioned)"]
      AN --> RF[revise_finding]
      RF -->|PM says 'that's it'| EC[end_call]
    end

    EC --> Hangup([Caller hangs up])
    Hangup --> CE[call-ended webhook
packages transcript + notes]

    subgraph MAESTRO["AFTER THE CALL — Maestro does all heavy lifting"]
      CE --> PC{"Haiku pre-check (NEW)\nescalate? reason? hand-back?"}
      PC -->|escalate: gas/fire| ESC[Alert PM immediately]
      PC -->|reason| RB[Reasoning brain\nemits ordered action queue\nfrom a closed vocabulary]
      RB --> AQ["Action queue (PMS-agnostic)\ngather → judge → dispatch → persist"]
      AQ --> RECAP[Recap SMS to PM]
    end

    RECAP --> YES{PM replies 'YES'}
    YES -->|confirmed| DISPATCH["Dispatch WOs (in-house immediate,\nexternal flag-gated) + stage charges"]
    YES -->|edit| RB

    GATE["🔒 Merge gate: real-call capture-only gauntlet\nreplays every real call through the live BRANCH agent (ADR-0048),\nasserts capture-only + clean hang-up"] -.guards.-> Walk

    classDef good fill:#1f5a2f,stroke:#57e07a,color:#fff;
    classDef gate fill:#1f3a5a,stroke:#57a7e0,color:#fff;
    class AN,EC,DISPATCH good;
    class GATE gate;

Turnover becomes a sibling domain of the maintenance Maestro (ADR-0053: "renewals/tours get sibling Maestros later… never one god-orchestrator"), reusing the same primitives, PMS-agnostic at the action layer (PMS-specific tools resolve under the hood).

7. Decision (locked principles)

1. The live call captures notes. Nothing else. No actions until after hang-up.
2. Fewest possible tools on the live agent — capture-only set: find_turnover, start_inspection, append_notes, revise_finding, end_call, skip_turn. The 6 action tools removed: send_summary, confirm_turnover, get_turnover_status, add_charge, dispatch_work_order, apply_charge_decision.
3. All heavy lifting is post-call, in Maestro — dispatch and charges happen only after the call, only after the PM's "yes."
4. No walk-agent change merges without passing the real-call capture-only gauntlet.
5. Enforcement is structural, not prompt discipline — the tool is removed from the belt; the prompt rule is a backstop, not the guarantee (ADR-0053).

8. Restore-and-harden plan

P0 — Strip the live agent to capture-only (through the pipeline; PROPOSED, held for owner approval).

• Remove the 6 action tools from agents/clara/lib/voice-agents/turnover-intake.config.json + their voice: blocks in src/lib/tools/turnovers.ts.
• Landmine: scripts/sync-turnover-intake.ts bindToolsToAgent is additive-only — it never removes tools (// NEVER removes a binding). Deleting tools from config will NOT unbind them from the live agent, and sync --check exits 0 (CI passes green) even though the live agent still has tools bound that are absent from the catalog. The same change must add a removal/reconcile path (diff live prompt.tool_ids against the catalog and unbind the extras) — still through CI/sync, never a manual curl PATCH.
• Branch-test on an ElevenLabs branch first (ADR-0048), not on the shared live surface.
• Add the prompt backstop: "On this call you ONLY capture findings via append_notes. Never dispatch, never charge, never confirm. When the PM says 'that's it', acknowledge and call end_call."

P1 — Build the real-call capture-only gauntlet and make it the merge gate (this is the thing asked for 10+ times).

• All 41 real harvested calls (+ Camellia) → faithful simulated-caller personas → live agent (branch) → assert capture-only (no action tool fires) + clean hang-up (ends on "that's it"). Model on l1-simulate.ts.
• Wire as npm run + CI gate on any agents/clara/lib/voice-agents/**, src/lib/tools/turnovers.ts, or walk-prompt change.
• See PR #2499 for the implementation and the scorecard proving it fails on the current (12-tool) agent.

P2 — Converge the post-call path onto Maestro (ADR-0061).

• Hang-up → Haiku pre-check (escalate / reason / hand-back) → reasoning brain → action queue (gather→judge→dispatch→persist) → recap SMS → PM "yes" → dispatch. Turnover as a sibling comms workflow; PMS-agnostic actions.
• Re-home add_charge / dispatch_work_order to the post-call action layer; the in-call versions cease to exist.

9. Secondary bugs (documented; fixes proposed in their own PRs)

• add_charge type bug — sends amount as a string; webhook rejects INVALID_INPUT: expected number. Every charge fails silently. Secondary because once the call is capture-only, add_charge does not run on the call at all; the coercion fix belongs to the post-call action layer. (Owner: "the main thing is this should have been easily detected" — the gauntlet would have.)
• Dropped inbound voice note — a PM voice-note reply (audio/amr MMS) to the recap is dropped: src/app/api/twilio/webhook/route.ts:224 passes only image/* media, then 400s on the empty body. Audio transcription already exists (ElevenLabs Scribe /v1/speech-to-text, used for post-transfer recordings; audio_transcribe action) — the MMS route just doesn't route audio into it. Fix + failing test in PR #2500 (ADR-0064).

10. Lessons

1. Test the layer that breaks. The regression is the live LLM's tool choice. Mocking tools and asserting on handlers measures the layer that was already fine. A harness that mocks tool returns is fine only if it asserts on tool choices.
2. Never put autonomous-action tools on a live customer-facing agent without a gate that proves they don't fire when they shouldn't (CLAUDE.md: ungating autonomous action is high-risk).
3. Green ≠ working. 13+ PRs merged green while the product got worse, because the gate was blind by construction.
4. Structural drift checks can't catch behavioral regressions. The nightly catalog⇄live drift check passed for turnover the morning after the regression — the action tools were legitimately in the catalog, so parity held. Parity ≠ appropriateness; only a behavioral gate (assert tool choices on real calls) catches "a capture-only agent acquired action tools." The additive-only sync still bites the fix: removing the tools from the catalog won't unbind them live, and the parity check won't flag the leftover binding either (it iterates catalog entries) — so the strip needs an explicit removal/unbind path plus an extra-tools check.
5. Keep the human's exact ask in view. "Real calls through a simulator as if I'm calling" was deferred for two days in favor of offline tests.

11. Consequences

• Positive: the live call is fast and safe (capture-only); the post-call path is the single place WOs/charges are created (matches ADR-0057 "PropFlow owns all dispatch"); regressions are caught by a real-call gate before merge.
• Cost: P0 needs the sync removal path built; P2 is real Maestro work (deferred until the SMS spine is proven — ADR-0053/rescope notes voice parity is behind SMS).
• Risk held: the live-agent strip is a production voice-agent change → proposed only, branch-tested, owner-approved before it ships.

12. References

Live agent agent_9601ksjwenzyecp9dpxxhad7a6xq; test bench property appfolio-45 / unit TEST-101 / turnover turnover_0a9b5eee…. Harness: evals/turnover-walk/**, scripts/simulate-walk-call.ts, scripts/voice-harness/drivers/l1-simulate.ts. Sync: scripts/sync-turnover-intake.ts. Incident: EL conv_1201kveyz963fcysbk7cfx22pf6t.