Clara leaked her internal reasoning — and two other people's names — in an email to a prospect
INCIDENT
Proposed fixes — pending Fede review
Aug 24, 2026, 9:02 PM · Camellia Apartments · email to prospect Casimir De Maigret · investigated Aug 25 morning
What the prospect received. Clara's reply opened with a sentence that was never meant to leave the building:
"This is Casimir's own message (not Sadie-Kate/Jace's on-behalf visit), so I'll treat it as ordinary."
That sentence is Clara's private decision-making, and it names two completely unrelated customers — Sadie-Kate Ball and Jace, a couple from an earlier tour. The rest of the reply was fine. All four safety checks on the turn passed. The debug widget that was built to show Clara's reasoning showed nothing for this turn.
The five-step causal chain (every step verified, not inferred)
- Aug 2–3 — a real on-behalf tour left a standing "expected visitor" note. A prospect (Jace) had his girlfriend Sadie-Kate tour for him. The system stored a hint on his inquiry: "Sadie-Kate Ball — Jace's girlfriend, touring on his behalf. Her phone: (…)". Jace has since signed a lease, but the hint stays live for 30 days regardless.
- The hint is whispered to Clara on every new leasing email at the property. The expected-visitor feature (ADR-0101) injects that note — names and phone number included — into Clara's context on every unlinked inbound leasing turn at Camellia, so she can recognize an expected visitor. It does not check whether the tour already happened or the deal already closed. So three weeks later, Casimir's unrelated inquiry turn still carried "Sadie-Kate, touring for Jace" in context.
- Aug 23 — a new instruction asked Clara to write an internal note inside her replies. PR #6159 ("Clara states her real reason") tells her to end every email/text reply with a machine-stripped marker:
[[WHY: one-sentence reason]]. The marker is removed before delivery and shown in the Decision Line widget. Review hardened every malformed-marker case (marker mid-text, truncated, duplicated) — but not the case where the note is written without the marker syntax at all.
- Aug 24, turn 2 — Clara skipped her thinking step and narrated instead. On the first turn that evening she reasoned about Sadie-Kate inside her private thinking block (captured in the trace) and the reply came out clean, with a well-formed
[[WHY]] at the end. On the second turn the model produced no thinking block (thinking is adaptive — the model chooses) and no marker — it wrote its deliberation as the first paragraph of the reply, in plain prose.
- No guard recognizes unmarked reasoning, so it shipped. The marker-stripper matches only
[[WHY: syntax (none present). The internal-prompt strip matches only scaffold phrases and tool names (none present). PII and fair-housing checks don't cover "names of other customers." The paragraph sailed through all four and was delivered.
Why the debugging widget was empty on exactly this turn
The Decision Line's reasoning card (PR #6189, shipped Aug 24) shows, in order of preference: Clara's stated [[WHY]] reason → her thinking summary → a reconstructed action log. On this turn there was no marker and no thinking block, so the card had nothing to show — and an absent marker raises no alarm (the critical alert only fires when marker syntax appears malformed). The one turn that most needed debugging is the one the widget is structurally blind to. Separately, the trace stores the system prompt but not the injected context blocks that ride in the message stream — which is why the trace showed no origin for "Sadie-Kate/Jace" at all.
What's real vs. assessed
| Claim | Status |
|---|
| The leaked sentence was delivered to the prospect | Verified (conversation record + trace raw output are identical; thread shows Delivered) |
| Sadie-Kate/Jace context was injected via the expected-visitor hint on Jace's lease-signed inquiry | Verified (hint record in prod DB; turn-1 thinking summary explicitly weighs "expected visitor Sadie-Kate Ball") |
| Turn 2 had no thinking block and no [[WHY]] marker; all guards passed | Verified (turn trace: raw model output, step list, stop reason) |
| The [[WHY]] instruction (day-old) is what nudged the model to narrate its decision in prose | Assessed, not provable — the instruction legitimizes decision-notes in the output stream, and the failure appeared the day after it shipped; but a no-thinking turn can narrate on its own |
Immediate cleanup already done (Aug 25 morning)
- Casimir's tour record corrected to a virtual tour (he's out of state and asked for a video call; Clara booked it as in-person). This stops the pre-tour text from sending in-person arrival instructions — a virtual tour sends none, and staff get a "place the video call" nudge instead. Audit entry written on the record.
- Still open, needs a human decision: Casimir was never sent any confirmation of the time (he asked "after 4pm MST or Friday"; the system silently booked 4:00 PM same-day). Someone should confirm the video call with him — and Clara owes him the promised parking-waitlist answer "tomorrow" (today).
Proposed fixes — pick per line (A/B/… choices)
1. Stop internal notes from ever reaching a customer
1A — Belt and suspenders: enforce the marker + add a third-party-name guard. (i) When a reply arrives with no [[WHY]] marker, log critical and run a cheap second-pass check on the first paragraph for deliberation shape ("I'll treat", "this is X's own message", references to rules/context). (ii) New outbound guard: scan reply text against the names of other people known at the property (persons/prospects not party to this conversation) — this catches the privacy half regardless of how the reasoning leaks. Would have caught this incident twice over.
1B — Move the reason out of the text stream entirely: have Clara return the reason via a structured field/tool call instead of inline markup. Cleaner long-term; bigger change to the reply loop.
1C — Force thinking on every turn (non-adaptive) so deliberation always has a private home. Cost increase on every turn; still doesn't guarantee no narration.
2. Stop stale on-behalf hints from haunting new leads
2A — Retire the hint when it's spent: exclude inquiries in terminal stages (lease_signed, lost, closed) from expected-visitor recognition, and stop injecting the free-text note's phone number into the prompt (Clara never needs the visitor's phone to greet them).
2B — Also shorten the hint TTL to 14 days. (Can combine with 2A.)
3. Make traces show where injected context came from
3A — Record injected context blocks in the turn trace (expected visitors, completed actions, etc. — the message-stream blocks), so "where did Clara get X?" is answerable from the trace page. This investigation required raw DB queries to answer it.
3B — Also surface "no reason captured" as a visible warning state on the Decision Line card instead of an empty card.
Companion finding from the same investigation (separate bug)
Every Camellia website-form lead has been silently dropped since Aug 22. The website contact form has always delivered leads as an email sent from Clara's own address to the Camellia mailbox. The Aug 22 fix that stops Clara from replying to her own notification emails (PR #6115, the Cara Urban loopback) skips any email from our own domain — which now includes every website lead. First real casualty: Raul Perez, Aug 25 12:42 AM (his email is in the mailbox under subject "Website contact for Camellia Apartments"; replying to it reaches him). Fix: exempt the website-form shape from the loopback gate (mirror the existing AppFolio "New Lead" relay exemption), or change the form's sending address.
Investigated Aug 25, 2026. Sources: prod conversation record + agent traces (both turns), prod DynamoDB (inquiry hint, tour record), email ingestion log, repo history (PRs #6115, #6159, #6189, ADR-0101).