Voice nightly: every caller, every question

Last run 2026-08-13 · LIVE · 1 call per box

Every night we phone Clara as ten different kinds of caller and ask each of them nine different questions. Each box below is one of those calls. It is green when the caller reached the right desk and got a straight answer with the right information in front of them.

This grid exists because of a real call on 11 August: someone who had already signed a lease rang up about collecting their keys, and was sent to the team that sells apartments — the one desk with no way to book a key handover. That call is one box on this grid. So is every other version of it we had not thought of yet.

The grid

What is broken

Boxes nobody has ruled on

These run every night and are reported, but they never fail the run — because we have not agreed what the right answer is. Each one is a decision waiting to be made.

• Caller we do not recognise → Where is my application? — Leasing owns the application funnel, but no routing rule names "application status" for a caller who is not on file. Leasing (the funnel) is the guess; message-taking is the plausible alternative.
• Caller we do not recognise → Where is my application? — No voice tool anywhere can read an application status, so the honest ceiling here is a promise to follow up. Whether that is good enough is an owner call.
• Caller we do not recognise → Keys and move-in day — Resident services holds the only key-pickup tools, but triage cannot send an unverified caller there — the identity gate says ask first. Expect one identity question, then resident services once they confirm.
• Caller we do not recognise → Something is broken — Maintenance can troubleshoot, but with no verified resident it cannot file anything. The honest ceiling is troubleshooting plus a handoff; nobody has ruled on whether that is enough.
• Caller we do not recognise → Renewing or leaving — Renewals will not find a lease for an unrecognised caller. Whether it should hand back to a person or take a message is undecided.
• Brand-new apartment shopper → Where is my application? — Same open question as the unrecognised caller: with no notes on file this caller is indistinguishable from one, and no rule names application status.
• Approved, lease not signed yet → Keys and move-in day — Approved-without-a-lease is still a prospect to the system, so the leasing fast-path fires — but resident services is the only desk with key tools. Whether an approved applicant should reach them early has never been decided.
• Approved, lease not signed yet → Keys and move-in day — Leasing can explain the sequence but cannot book anything. Whether that is the right answer or a coverage gap is an owner call.
• Lease signed, moving in soon → Building rules and amenities — Resident services owns everything about their move-in, so that is the guess — but a pure amenity question has no rule of its own for this caller, and leasing is closed to them.
• Lease signed, moving in soon → Building rules and amenities — Depends on the routing question above.
• Lease signed, moving in soon → What do I owe? — An incoming resident has an empty resident id, so the balance tool has to re-resolve them server-side. Whether it returns anything useful before move-in is unverified.
• Resident past lease end (month-to-month) → What does my lease say? — The voice context has no month-to-month marker at all — the webhook sends the same returning-resident header as any other resident. Whether Clara can tell them the truth here has never been tested.
• Resident past lease end (month-to-month) → Renewing or leaving — Nothing in the voice context tells Renewals this resident is month-to-month. What it can honestly say is untested.
• Moved out → What does my lease say? — A closed account has no live lease record to read out, so this probably belongs with a person — but no rule covers a former resident asking for paperwork.
• Moved out → Prices and tours — Triage rule 5 is closed only to CURRENT and INCOMING residents, so leasing looks open to a former one — but no rule says so, and this is exactly the ambiguity worth surfacing.
• Moved out → Prices and tours — Depends on the routing question above.
• Moved out → What do I owe? — Deposit refunds are a money question about a closed account and read like rule 6 work — but the existing former-resident scenario only bans the wrong tools, it never pinned a destination. This cell is the open question.

What changed since last night

• Approved, lease not signed yet → Where is my application?: Working → Broken

Everything else that ran tonight

What this run did not cover

• Nothing here places a real phone call or writes to any property: every call is simulated in text with all tools mocked, against the test property only. Real-audio and end-to-end write paths are covered by separate harnesses.
• Production data checks that need AWS access are not run here — this workflow is deliberately scoped to the ElevenLabs key alone.