Why Clara's phone replies got longer, what was measured, what was ruled out, what changed (nothing yet), and the decision on how to fix it.
2026-08-28 · Camellia production voice line · investigation complete, fix designed & awaiting go · author: Claude (Fede's session), data from 3,018 real call transcripts + live ElevenLabs config
Clara's spoken answers to prospect questions roughly doubled between June and August — because in mid-July we gave her the facts and told her to say them, without telling her to answer only what was asked. The model, temperature, and ElevenLabs settings did not change and were ruled out with live reads. Nothing has been modified in ElevenLabs during this investigation (read-only throughout), and the designed fix has not shipped — a before/after harness exists and should run first. Decision options at the bottom.
On the 2026-08-27 9:11 PM call (conv_5101m135k2szedwbv2c90hzde26x, unknown caller, Camellia), a caller asked "can you tell me what utilities normally run?" — a cost question — and got a 94-word spoken monologue covering the billing provider, account setup with the electric company, and portal mechanics. Read aloud, that's ~38 seconds of uninterrupted talking. The same question class over time, same line:
| Date | Reply to a utilities question | Words |
|---|---|---|
| 2026-03-09 | "I don't have specific information about which utilities are included… would you like to schedule a tour?" | 28 |
| 2026-05-21 | "I don't have the specific utility details on hand, but the leasing team can walk you through…" | 50 |
| 2026-08-05 | Full recitation: who pays what, third-party biller, ~$6.50 admin fee, electric setup | 72 |
| 2026-08-27 | Full recitation plus typical dollar ranges and the "not a flat fee" caveat | 94 |
Worst August offenders: a 130-word answer to a move-in-cost estimate (Aug 25) and a 212-word answer to a lease-discrepancy question (Aug 26).
Method: all 4,038 ElevenLabs conversations in retention (Feb 18 → Aug 28) were listed; the 3,070 on production and sandbox agents were fetched in full (98.3% success), agent-turn words counted by whitespace split, grouped by ISO week. Sandbox ([WILLOWS]) and bench/eval-robot traffic excluded from the trend. On the clean leasing lane (Triage + Leasing + the pre-split agent, scripted outbound excluded), and on the stricter prospect-topic subset (the caller is actually talking tours/pricing/units/utilities):
| Period | Leasing lane — mean words/turn | p90 | Prospect-topic subset — mean | p90 |
|---|---|---|---|---|
| Feb–Apr baseline | 19–27 | 37–60 | 19–27 | 38–59 |
| June trough | 10.6–14.1 | 20–30 | ~12–15 (thin weeks) | 13–36 |
| Aug (W32–W35) | 20–23.5 | 41–57 | 24–29.4 | 49–61 |
On pure prospect calls, August is the highest sustained level of the year — ~25–30% above the winter baseline and ~75% above June, with the final measured week (W35) the peak of the whole series. The June trough is partly a traffic-mix artifact (scripted renewal-outbound season); the August level is not — it holds on the clean cut. The jump begins exactly in the weeks of July 6–19.
Primary: knowledge arrived without answer-scoping. Between July 9 and July 22 a burst of changes moved real property facts into the voice agents' context and told them to answer directly instead of deflecting: building-announcement awareness (Jul 9), amenities/neighborhood facts into the leasing prompt (Jul 14), direct amenity answers at the front door (Jul 14), a full utilities fact block (Jul 18), mandatory pre-tool speech (Jul 20), context-aware greetings (Jul 22). Each is individually good — Clara answering beats Clara stalling. But the utilities rule reads "say plainly what's included in rent and what the tenant pays" — both sides of the ledger, every time — while the neighboring LEASE ANSWERS block already carries the correct discipline: "LEAD with the single fact they actually asked for; at most ONE dollar amount and ONE date in a reply; NO lists… offer to text the full breakdown." One block got the spoken-delivery contract; the others didn't. Clara does what she's told.
Secondary: rule accretion. The Triage prompt doubled (277 → 506 lines) between July 20 and August 13 as incident fixes each added a mandatory behavior; long answers like the 130-word cost estimate are Clara dutifully stacking every required element in one turn.
Mechanism note — the content is data, not code. The fact blocks are injected per call from the property-knowledge record in the database ({{utilities}} and siblings, filled at ring time). As the team enriches a property's knowledge, Clara automatically has more to recite — so without a scoping rule this compounds at every property we onboard.
Attribution correction (important for who gets fixed): the verbose turns come from the Leasing agent, not Triage. Per-turn agent ids in the transcripts show Triage obeying its 2026-07-21 routing rule — handoff line, transfer, done — and ElevenLabs simply files the whole call under the first agent's name, which made per-agent stats misleading until read at turn level. The fix targets the Leasing prompt and the shared injection rules.
claude-sonnet-4-6; temperature 0.0 on all inbound agents, unchanged.| Agent (production) | LLM | Temp | Prompt chars | Reply token cap |
|---|---|---|---|---|
| Clara — Triage | sonnet-4-6 | 0.0 | 79,762 | unlimited (−1) |
| Clara — Vendor Calling (Out) | sonnet-4-6 | 0.3 | 79,418 | unlimited (−1) |
| Clara — Leasing | sonnet-4-6 | 0.0 | 67,335 | unlimited (−1) |
| Clara — Maintenance | sonnet-4-6 | 0.0 | 31,887 | unlimited (−1) |
| Clara — Renewal (Outbound) | sonnet-4-6 | 0.85 | 30,622 | unlimited (−1) |
| …7 more (Renewal, Handyman, Resident Svcs, Unknown Caller, Turnover, Emergency, Escalation) | sonnet-4-6 | 0.0–0.85 | 2,813–28,097 | unlimited (−1) |
Turn-taking config is healthy: 7s turn timeout (in ElevenLabs' recommended 5–10s band), eagerness "eager", soft-timeout filler at 4s with LLM-generated lines, interruptions enabled with backchannel-ignore ("uh-huh", "mm-hmm") — a caller can cut a monologue, which is damage control, not a fix. TTS speed 1.0, max call 10 min.
Target set by Fede: ≥30% shorter info answers without sounding rushed or dropping key details. The designed change: (1) factor the LEASE ANSWERS spoken-delivery contract out so it governs every injected fact block — lead with the asked fact, ≤2–3 short sentences, one dollar amount and one date per reply, offer depth instead of delivering it ("want me to text you the breakdown?"); (2) set the per-turn reply cap from unlimited to ~150–200 tokens as a backstop; (3) sync the two drift findings both directions.
| Option | What happens | Trade | |
|---|---|---|---|
| A · Recommended | Measure, then ship | Run the built harness: replay the real caller questions against a temp cloned agent carrying the fix (ElevenLabs simulate API — no phone legs, live agents untouched), score before/after against the ≥30%-shorter / no-dropped-facts / not-clipped contract, then ship on a passing report and re-run the weekly trend a week later. | ~half a day before the fix is live; highest confidence. |
| B | Ship now, measure after | Land the scoping rule + cap directly (repo → sync), watch the conciseness scorecard on live traffic. | Fastest relief; risk of an unmeasured tone/coverage regression on live callers, which is how this problem started. |
| C | Hold | Keep current behavior; scorecard continues watching. | Prospects keep getting 30-second monologues; compounds with every property onboarded. |
Separate but adjacent, not blocked on this decision: sync the Leasing live↔repo drift, commit the untracked Renewal-Outbound text, Triage prompt diet round 2, and add words-per-turn (mean + p90, leasing lane) to the voice scorecard so this trend is watched, not re-noticed.
Fede picked Option A; the harness ran the same day. Setup: two throwaway clones of the live Leasing agent in ElevenLabs (live agents untouched) — one byte-identical control, one carrying a fix variant — each replied to the 8 real caller questions × 3 reps through the simulate API (no phone legs, all tools mocked), grounded in the real Camellia knowledge blocks rebuilt from prod through the production injection builder. Deterministic scoring; no judge in the loop.
| Arm | Mean words/answer | vs control | Worst lanes | Clipped replies |
|---|---|---|---|---|
| Control (today's live prompt) | 71.7 | — | fees 108–124w · utilities 78–100w | 0 |
| Variant 1 — scope rule in prompt + 300-token cap | 68.2 | −5% | fees unchanged (108–124w) | 0 |
| Variant 2 — v1 + restructured block content (headline fact first, provider/setup mechanics behind an explicit "only if asked" fence) | 58.1 | −19% | utilities −39% (~92→~56w) · fees −49% on 2/3 reps (~115→59w) | 0 |
What the numbers say. Adding another brevity rule to a 67,000-character prompt does almost nothing — the prompt already carries three such rules and the model recites the block anyway. The lever is the injected content itself: when the utilities block leads with the $40–50 ballpark and fences the Xcel/Conservice mechanics behind "only if the caller asks," the recitation stops — the injected facts themselves are only moved, never deleted, and nothing scored as clipped. Correction (same day, evening): this section originally said "nothing dropped"; a per-fixture pass on the same run found variant 2's replies omitted one required fact on the compound move-in question in 2 of 3 reps (control: 0 of 3) — an authoring error in the first write-up (the data was in the run), not a scorer change. Every word-cutting variant tested since shows the same tension on that lane: a two-fact question has a physical word floor. The full tunability matrix (§8c, below) carries the paired per-fixture gate that supersedes this section's per-lane claims. The remaining fat is now characterized: unasked concession mechanics on pricing answers, stall openers ("Sure, let me get that…") that a same-breath rule already forbids, and one contract that's arguably over-strict (pet policy's three dollar figures are the legitimate answer). Those are the overnight loop's next variants; 30% is in reach.
Ship path this implies: apply the restructuring in the injection renderer (buildLeasingContextInjection utilities/fees rendering — one code change, ships to every property), keep the prompt scope rule as belt-and-braces, let the overnight harness iterate on the residual lanes, and re-run the weekly trend after a week live.
Fede's steer: "we should be able to tune how the voice agent behaves instead" of restructuring injected facts — prove tunability before betting on content-shaping, and "don't over engineer." Five prompt-side variants ran on the same harness (8 real caller questions × 3 reps each, identical real prod grounding, live agents untouched), against the control and the held content-restructure:
| Arm (one factor each) | Mean words | vs control (71.7) | Newly dropped facts (vs control, per lane) | Replies leaking unasked mechanics |
|---|---|---|---|---|
| Scope rule in prompt + token cap (v1) | 68.2 | −4.8% | utilities (1 lane) | 5 |
| Prompt diet (67K→46K chars) + same rule (b2) | 71.0 | −0.9% | move-in | 6 |
| Scope dial colocated beside every fact block (b3) | 63.6 | −11.3% | move-in | 6 |
| 3 positive exemplars, no rules (b4) | 67.7 | −5.5% | move-in | 6 |
| Rule + cap + Sonnet 5 behind the agent (b1 — compound arm) | 55.7 | −22.3% | cost-estimate + move-in (money lanes) | 5 |
| Content restructure, held (v2 — headline fact first, mechanics fenced) | 58.1 | −18.9% | move-in | 0 |
| EL-native KB, RAG mode — INDEXED re-run (kb5a — messy content as-is, retrieval per turn) | 61.7 | −14.0% | none — the only arm with a clean fact sheet | 5 |
| kb5a's first run (−28.3%, 12 fact-missing replies, 3 stalls) measured an unindexed document and was struck as invalid — EL uploads never index automatically; the workflow now indexes and polls to "succeeded" before any measurement (finding credited to the parallel POC session). | ||||
| EL-native KB, full-context mode (kb5b — messy content as-is, platform-managed) | 61.5 | −14.2% | move-in (1 rep) | 4 (= control) |
The kb-native answer (Fede: "we've never explored an eleven labs native property knowledge solution"). Both modes ran with the messy source content uploaded byte-for-byte as-is, prompt carrying zero facts (67K → 56K chars, latency-positive per EL's own guidance; clones materialized through the CI/CD-only path). RAG mode, properly indexed, is the day's best behavior result: −14.0% words with the only clean fact sheet of any arm — zero lanes dropping facts vs control, and it improved the two lanes control fails today (tour availability, and it even passed the physical-floor move-in question in 2 of 3 reps, both dollar facts intact under 50 words). Five contract passes, the most of any arm; zero clipped. The mechanism makes sense in hindsight: retrieval naturally does what the manual restructure does by hand — the model only sees the passage relevant to the question, so there's no messy block to recite. Cost side: simulate-level median 14.1s vs control 12.8s (+10%, consistent with EL's documented ~250ms/turn retrieval overhead; the live-TTFW round remains the binding latency test). The full-context mode (kb5b) trades slightly more words for zero retrieval risk. Both native-KB modes now beat every prompt-side variant, and the restructure keeps the outright word/leakage lead. A footnote on process: kb5a's first published verdict ("RAG disqualified") was an artifact of measuring an unindexed doc — struck, corrected, and the indexing step is now enforced in the workflow. Full-context mode is the real finding: −14.2% with zero clipping, the most contract passes of any arm (3), a fact profile matching control everywhere except the physical-floor move-in lane, and leakage no worse than control — the second-best arm overall, beaten on words and leakage only by the held restructure. Simulate-level timing was flat (13.3s vs 12.8s median; non-binding — the live-TTFW gate still decides latency). If the restructure ships as the content story, kb5b is the strongest candidate for where the content lives: platform-managed full context with an 11K-char lighter prompt, pending the robot-call TTFW round.
Control leaks unasked mechanics in 4 of 24 replies and misses required facts on two lanes (all-fees, tour-availability) today — b1 and the rules arms actually improved those two lanes. Sonnet 5 was accepted by ElevenLabs and executed all 24 simulations.
Reading. (1) Behavior-tuning on the current model tops out around −11% (colocated dial, the best pure-prompt arm) and no prompt-side variant stopped the provider-mechanics leakage — including a 32%-lighter prompt and the exemplar approach the platform docs recommend. (2) The content restructure is the only arm with zero leakage, at −19% — consistent with industry practice (headline-first / tiered knowledge authoring is documented standard at Retell; ElevenLabs' own guidance puts prompts at ≤~2K tokens vs our ~17K, so slimming is also latency-positive). (3) Sonnet 5 follows rules dramatically better (−22%) but dropped required money facts in this compound configuration and would need the full triage/escalation voice suites plus the live-latency gate before any consideration. (4) The move-in lane drops a fact in every word-cutting variant: a two-fact question has a physical word floor — the fix is per-lane floors in the contracts, not cleverer variants. (5) Prior art honored: the prompt was already dieted once (PR #4275, Jul 21, 14.1K→11.4K est tokens) and has regrown +49% in five weeks; per-question retrieval was tried before and is documented at ~+250ms/turn — ruled out on the latency constraint.
Fede's go: "real calls, real evals, multi turn including tool calls", criterion "a scalable solution". Six arms each took the same three multi-turn robot-call scripts into the bench line (utilities multi-turn, fees/move-in, tour-booking with the live booking webhook) — 21 real calls total, the line remapped per arm via one-line config PRs and restored after (PR #6511). TTFS = time-to-first-sentence per agent turn from the platform's own call metrics: the time-to-first-word proxy on a real call.
| Arm | Mean words/reply | p90 | TTFS median | TTFS p90 |
|---|---|---|---|---|
| Live fleet today (reference; triage→leasing topology) | 36.8 | 78 | 1.89s | 2.14s |
| Control clone (prompt-injected knowledge) | 33.2 | 66 | 1.56s | 1.86s |
| Messy KB · retrieval (kb5a) | 35.5 | 72 | 1.64s | 2.04s |
| Messy KB · full-context (kb5b) | 31.9 | 63 | 1.59s | 2.00s |
| Distilled KB · retrieval (kb6a) | 35.3 | 66 | 2.58s | 12.08s |
| Distilled KB · full-context (kb6b) | 34.6 | 69 | 1.47s | 1.84s |
What the live round settles. (1) kb6b wins the binding metric: distilled knowledge in the platform's native KB, full-context mode, is the FASTEST arm to first word — median and tail — beating even the control, while carrying the scalable story (automatable distiller, 11K-char lighter prompt). (2) Retrieval mode is rejected on the tail: kb6a hit a 12-second p90 first-word stall on one retrieval turn — exactly the added-latency class the constraint forbids; a tail that shows up in 3 calls will show up for real callers. (3) Placement alone does not cure the recitation: the utilities cost question got the provider dump in every arm on real calls; fees passed everywhere. The conciseness cure remains the answer-behavior dial, now iterated cheaply on this harness against the winning architecture. (4) A real product bug rode along, arm-independent and now multiply-confirmed: the tour tool path claims a booking while no Tour row lands (schedule→reschedule sequence from a caller with prior cancelled tours) — filed separately from this decision. The decision itself — adopt kb6b, hold, or widen the sample — is presented as an A/B/C pick in messy-input-is-on-us-2026-08-28, awaiting Fede.
Caveats stated plainly: three calls per arm; ~100ms medians are within noise; the kb6a tail spike is a single turn — which is what tail risk is. Round mechanics: every line remap was a reviewed one-line PR (#6485→#6511) through the CI/CD-only guard; two silent-fallback classes (unindexed RAG docs, credential-less seed-data units in CI) plus three clone-wiring gaps (personalization enable, transcript webhook, override permissions) were caught by content-level verification before any arm raced dishonestly — the harness now enforces all five.
Corpus: 4,038 conversations listed, 3,018 transcripts fetched and cached; trend computed per ISO week × agent group; examples quoted from real transcripts with conversation ids retained as handles (no caller PII in this page). Live config: GET-only reads of all agents, KB, and turn config on 2026-08-28. Change timeline: git history of agents/clara/lib/voice-agents/* since 2026-05-01. Platform guidance: ElevenLabs prompting guide, LLM cost docs, conversation-flow docs (fetched 2026-08-28). Attribution correction verified from per-turn agent_metadata.agent_id in the Aug 27 transcript. Related decision doc (email/SMS effort, separate workstream): effort-sweep-email-sms-2026-08-27.