Turnover PM-Flow RCA & Robo-Test Findings
Root cause of the confirm loop (verified, not inferred)
Evidence: conversation 0682c07f tool rows (three consecutive honest replies audited as [TURNOVER-GUARD-BLOCKED], each with a successful forward_to_property_manager and an errored dispatch_work_order: identity_mismatch); Person pers_2f188e8b has PROFILE + phone claim but zero OCCUPANCY# rows. The bench turnover is the vendor-calling rig's fixture — left untouched by design; it stays the "wedged state" test asset.
Findings from the robo run
| # | Finding | Severity | Status |
|---|---|---|---|
| 1 | Fabrication guard swallows honest dispatch-failure replies (negation filter was branch-scoped) and its fallback routes the PM to the wrong tool ("reply yes" → confirm, which can't retry a dispatch) | High — PM trapped in a loop | Fixed — PR #4883 (merged) |
| 2 | Edit-honesty guard blocked a truthful reply about a successfully created WO (WILL-002118) because Clara named it by id instead of the words "work order" — PM was told "nothing's been updated" about a real WO | High — truth inversion | Fixed — PR #4888 (merged) |
| 3 | Edit-honesty guard rewrote every honest walk-note logging reply ("all three are logged on that walk") into charge-edit decline copy — append_notes success had no exemption | High — every walk turn affected | Fixed — PR #4893 (merged) |
| 4 | find_turnover misses prefixed unit labels: query "17" ≠ stored "WLH-17" (rows store both shapes on one property) — a three-item walk had nowhere to land | Medium | Fixed — PR #4891 (merged; reworked onto the canonical exact-first / unique-or-nothing matcher after bot review) |
| 5 | Vendor fuzzy-match misses punctuation: "Miracle Cleaners QA" ≠ "Miracle Cleaners (QA)" — WO silently created unassigned | Medium | Fixed — PR #4889 (merged; delegates to matchVendorCompanyByName + archived-vendor filter) |
| 6 | Injected turnover context tells the PM only ONE turnover is open ("the only turnover I have open here is 101") while three were open — find_turnover knows better when asked | Medium | Open — design gap |
| 7 | confirm_turnover is a hard NOOP once tasks exist (already_scoped): a second walk's new findings never mint tasks/WOs, while the copy claims "the work is underway" — new items silently dropped unless the PM names a vendor per item | High — silent work loss | Open — design gap (needs delta-scoping) |
| 8 | No cancel/change-vendor tool: "don't call them" leaves the VendorCallSession sitting in awaiting_approval; Clara can only say "don't approve it" and escalate (matches the known #2445 intent-coverage gap) | Medium | Open — known gap, now with a live repro |
| 9 | Unrelated: main's npm test was red repo-wide (nightly alert copy map missing the new publish-eval-runs job) | CI blocker | Fixed — PR #4886 (merged) |
PM-perspective flow results
| Flow | Result | Notes |
|---|---|---|
| 1 · Walk → plan → approve → WO → session | PARTIAL | Walk logging + per-item dispatch work end-to-end (WILL-002119/20 + two awaiting_approval sessions to the QA vendor, verified in DDB) — but only via PM-named vendors; the confirm path minted nothing (finding 7), and two truthful replies got guard-rewritten en route (findings 2, 3). |
| 2 · Multi-item walk, mixed trades | PARTIAL | Three items logged in one message after the unit-label miss (finding 4); per-item vendor split handled correctly, vendor-less item explicitly flagged back to the PM. Good behavior once past the lookup. |
| 3 · Unit-mismatch correction | PASS | Wrong-unit report (201 vs 101 in the original wedge; 12 vs 101 in this run) is flagged before any write, and an explicit correction re-binds via find_turnover to the right turnover. |
| 4 · Add item after plan consumed | FAIL | Finding 7: the carpet item became a walk note but no task/WO; "work is underway" copy claims otherwise. Escape hatch = name a vendor (dispatch path). |
| 5 · Decline / change vendor at approval | PARTIAL | Honest and safe: correctly states the call was only queued, escalates the WO cancel (no tool), creates the replacement WO honestly-unassigned. But the stale session can only be ignored, not rejected (finding 8). |
| 6 · Wedged-state recovery | PASS (live, post-deploy) | Re-ran the original wedge repro after #4883 deployed: Clara's honest reply now survives verbatim — "failed to dispatch again — same identity error… no work order exists… I've forwarded it" — no guard rewrite, no "reply yes" loop, and she correctly isolated the issue to unit 101's data ("dispatches on 12 and WLH-17 went through fine"). |
What shipped
| PR | What | State |
|---|---|---|
| #4883 | Fabrication guard: clause filter on every evaluation + error-aware fallback + identity-mismatch detail threaded into dispatch errors | Merged |
| #4886 | Nightly failure-alert copy map: publish-eval-runs entry (unbreaks npm test on main) | Merged |
| #4888 | Edit-honesty guard: dispatched-WO refs exempt truthful created-WO replies (clause-scoped after a 5-round review loop) | Merged |
| #4889 | Vendor match via canonical matchVendorCompanyByName, archived vendors excluded | Merged |
| #4891 | find_turnover unit matching via matchOccupantByUnit (exact-first, fail-safe on ambiguity, scoped before matching) | Merged |
| #4893 | Edit-honesty guard: sentence-scoped walk-note exemption | Merged |
Open design gaps (not fixable in a guard patch)
Second-walk delta-scoping (finding 7). confirmTurnoverScope treats tasks.length > 0 as "scoped forever". Idempotency for redelivered confirms is right; a whole-turnover NOOP is not. Proposal: derive candidate tasks from the current condition-report projection and mint only the ones not already covered by an existing task — same idempotency, no silent drops. Needs its own ADR-sized look at the projection/task identity rules.
Context injection names one turnover (finding 6). The PM conversation binds the most-recently-touched turnover and speaks as if it's the only one. Cheap improvement: the injected block should enumerate open intake-stage turnovers for the property (unit + tenant), so a walk on another unit routes without the PM having to argue.
Session/WO cancel tools (finding 8). "Don't call them / cancel that WO" has no tool; the rejectOutreach transition exists server-side but nothing LLM-callable reaches it. Already on the intent-coverage list (#2445); today's run adds a live repro.
Also produced
A full dossier of prior turnover-workflow ADRs, PRs, and the outcome-flow code map (including the edit_recorded/declined type-union asymmetry behind two earlier incidents) is on the mini at ~/agents/001/TURNOVER-DOSSIER.md.