Prompt change in review (PR #5755) — live 0-of-20 proof runs after merge
Three asks, one bar:
One carve-out stays sacred: if the caller asks for the price again ("sorry, what was that?"), they get the real numbers — every time.
The overnight fix stopped Clara repeating the special offer — that half held in all 10 live test calls. But 5 of 10 live calls still re-read the identical price range on the caller's second question. Root cause: the old instruction didn't ban the repeat — it suggested a script for it:
"same range I mentioned — the studios start at twelve-hundred"
That example models both failure shapes at once: the robotic self-quote, and a partial price re-read. When we ran the old prompt against the new stricter test, it failed 0 of 3 — producing that recap phrasing verbatim.
| What's graded | Old prompt | New prompt |
|---|---|---|
| Second answer repeats no price and uses no recap phrase | 0/3 — says "same ranges I mentioned" verbatim | 3/3 |
| New multi-turn brevity conversations (3 cases × 3 runs) | pass at temp 0 (kept as regression pins) | 9/9 |
| Over-tightening guards (special still lands; re-asks still answered) | — | 12/12 |
| Transfer speech unaffected by the brevity tightening | 3/3 | 5/5 |
| Full 33-conversation suite | — | 31/33 (the 2 misses fail identically on the old prompt — pre-existing, unrelated) |
Passed — zero repeats in 20 calls. The change merged and synced to the live phone agent on Aug 14. The same test that measured 5-of-10 repetition before the fix was then run 20 times against the updated agent (two batches of 10): 20 of 20 clean — no repeated price pitch, no recap phrasing, and no collateral damage (the move-in special still lands, and callers who ask again still get the number).
Repro: eval-voice-feel-subscription.ts --only item21|item28|item29|item30 (bridge, subscription); live: comprehensive-run.ts --scenario=reg-leasing-no-double-studio-pitch --repeat=10. Bug card: Trello BOuFORRt.