Full audit for the two #alerts this morning: which original piece of work caused each drift, why it happened, why it's not fixed in code yet, and what's still missing so it can't quietly happen again.
Written 2026-08-28 · read-only investigation, four research passes plus two independent verification passes · nothing in this doc has been applied to production
Both alerts trace back to the same root cause: nobody is currently pushing every kind of update from the code to Camellia's live phone agents. One bug caused a pile of duplicate tools; a separate, deliberate pause caused a real wording fix to never reach one call script. Both fixes already exist and are tested — they're just sitting there.
Camellia is the one property with real paying tenants on the phone with Clara. On Aug 23, by Fede's explicit order ("no voice changes in Camellia production until I test everything"), the automatic pipeline that pushes code changes onto Camellia's live agents was deliberately switched off — documented in the code and the drift checks adjusted to match — while the Willows test fleet was built out. Since then, updates to Willows ship automatically on every merge. Updates to Camellia wait for the hand-run job. What was never scheduled is the "un-freeze once tested" step — so approved fixes have accumulated for five weeks. The freeze worked as designed; it just has no expiry.
That gap produced two different symptoms this week:
The path from a code change to a caller's ear, and back to the nightly checks. Solid black = happens automatically today. Dashed orange = only happens if a person remembers to run it by hand. Dashed red ending in a mark = the path used to exist and was cut.
One correction the checks themselves needed: because Camellia is frozen, two of its drift checks (wording and turn-taking) were changed to compare the live agent against a snapshot taken at freeze time, instead of against the current repo. That stopped them from permanently crying wolf — but it also means they can no longer catch "a real approved fix never shipped," only a rogue dashboard edit. That's part of why alert #2 wasn't caught earlier by that particular check; it was caught by the separate wording-specific check instead.
| Property / agent | How it gets updated | Last time it actually happened |
|---|---|---|
| Willows (test property, no real tenants) | Automatically, every time code merges | Same day as every merge |
| Camellia (real tenants) — everything except phone routing | Someone has to run a manual job and type an exact confirmation phrase to prove they mean it | Not run once in the last 30 days |
| Camellia — which agent answers which phone number | Automatic, and the only lane that also fixes itself overnight if it drifts | Checked, and repaired if needed, every night |
| Renewal call script (all properties) | Manual button only — see incident 2 below | Aug 15 |
| Vendor-outreach agent and the after-hours emergency-relay agent | No update path exists at all. Merging a wording change does nothing to these two live agents — the only automatic thing running against them is a check every 6 hours that notices they're out of date and complains. Nothing then fixes it. | N/A — has never auto-updated |
Phone-number routing is the one lane built the way every other lane should be: it pushes automatically and repairs itself overnight if it ever drifts, with nobody having to notice or act. Every other lane either needs a person to remember a manual step, or has no path to production at all. That's the single clearest thing to copy elsewhere.
Over the last 30 days: the nightly check that compares every live agent against the repo failed on 7 separate nights. The 6-hourly wording-mismatch check failed 6 times (3 of those were today, for incident 2 below). The automatic push-to-Willows job itself failed 3 times on Aug 26 — one broken tool wiring sat live and wrong for roughly 19 hours before the next nightly check caught it, because nothing else was watching in the gap between checks.
Both write-ups live as comments in the code itself, next to the fixes that came out of them — the same place this week's two incidents will end up once they're closed.
| When | What happened |
|---|---|
| 2026-07-10 | A code change adding two new call-handling agents at once exposes an older bug: the tool-setup script doesn't remember what it already created earlier in the same run, so it mints one duplicate tool per agent. Cleaned up at the time. |
| 2026-07-19 | A throwaway test copy of the leasing agent is built the wrong way, creating 15 stray tools. The next full-fleet sync then grabs the newest copy of each tool and re-points 7 live agents onto the stray copies. Caught and fixed same day. |
| 2026-08-22, 6:16am | Someone runs the cleanup tool for real. Result: 64 tools, 1 duplicate name — the last confirmed-clean state. |
| 2026-08-22 – 08-23 | Three unrelated code changes merge (none touch the tool list on purpose). Each one automatically triggers the fleet-sync job, which is still writing to Camellia at this point. Each run's "does this tool already exist?" check quietly starts missing real tools once the list passes 100 — and creates a new copy instead of reusing them. More duplicates push more real tools past the same limit, so the next run misses even more. It snowballs. |
| 2026-08-23, ~6:20am | A change meant to protect Camellia while a second test property (Willows) is being built out splits the pipeline by property. From this point on, the automatic push-on-merge only targets Willows. Camellia is frozen for every surface except phone-number routing. The same change is where the renewal call-script push gets left out of the automatic path — logged in the code, in plain words, as "paused, not silent." |
| 2026-08-24, 6:15am | The nightly health check catches the tool pile-up for the first time: a false "missing tool" alarm on the call-transfer handler (the tool was there, just hidden past the 100 mark) plus 20 groups of duplicate names. |
| 2026-08-24 – 08-27 | Tool counts keep climbing as the underlying counting bug keeps running its course; the nightly check flips red and green as the numbers shift. |
| 2026-08-27, 9:41am | The root cause is fixed everywhere it appears in the code (8 different spots were all only reading the first page of the tool list; all 8 now read the whole thing). The fix explicitly does not clean up the duplicates already created — mutating live customer-facing config unattended is called out, in writing, as something that needs a person to approve. |
| 2026-08-28, 3:23am | A wording fix to the shared call script — making sure Clara never promises to text something without the text actually going out — merges and reaches every Clara voice agent automatically, except the renewal-outbound one, because that path is still paused. |
| 2026-08-28, 3:25am | Two minutes later, the automatic check built for exactly this catches the mismatch and turns red. It has stayed red on every run since. |
| 2026-08-28, 9:12am | Someone runs the duplicate-tool cleanup tool again — in preview mode only. It prints the exact fix (37 duplicate groups, 92 tools to remove, precise before/after counts per agent) but doesn't apply it. These numbers match this morning's alert exactly. |
| 2026-08-28, today | This report is written. Both fixes are ready. Neither has been applied. |
Before creating a tool, the sync script is supposed to check "do I already have one of these?" by pulling the full list of tools from ElevenLabs. The bug: it only ever asked for the first page of that list — the first 100 results — and ElevenLabs hands results back a page at a time. Once the workspace had more than 100 tools in it, any real tool sitting on page two became invisible to that check. The script saw a tool that already existed, concluded it was missing, and created a brand-new copy. Each new copy pushed one more real tool past the 100-item cutoff — so the very next run missed one more tool, and created one more duplicate. It fed on itself.
Imagine a filing clerk who only ever checks the front of the cabinet before making a new folder. Once the cabinet has more than a drawer's worth of folders, everything behind that point looks "missing" to him — so he keeps making duplicates, which push more folders further back, which makes him miss even more next time.
Three routine code merges between the evening of Aug 22 and the morning of Aug 23 — a booking-consent fix and two sound-quality tweaks for the test property — each automatically triggered the fleet-sync job as a side effect of merging to the main branch. None of the three changes touched the tool list on purpose; running the already-broken reuse-check three times in a row was enough on its own. No person ran this by hand, and the separate manual-only Camellia sync tool has never been run at all.
The same page-one-only bug also lived inside the nightly health check that's supposed to catch exactly this kind of drift. So for the whole window this was happening, the safety net watching for it was reading the same truncated list as the tool that was breaking things — it just hadn't crossed the 100-item line yet either. Both were fixed together on Aug 27.
We found no evidence of it. The scariest single alert — a tool that looked completely missing from the call-transfer handler — turned out to be the exact same page-one blind spot: the tool was there, just past the cutoff. Every real, non-false-alarm duplicate we found is agents ending up bound to the same tool twice under one name, which the team has hit before (mid-2026) and every writeup of it says the same thing: this makes a nightly check go red, it does not stop Clara from doing her job on a call. We're stating that as "nothing found," not as a guarantee — nobody has written a detailed after-action report on this exact shape of duplicate before, so it's worth someone's eyes before it's closed out as fully harmless.
Separately: a real chunk of the 92 tools flagged for deletion today aren't from this bug at all — they're leftovers from an earlier internal experiment (comparing two different AI voice models against each other) that was flagged back on Aug 24 as "safe to delete, waiting for a go-ahead," and simply never got deleted.
The renewal call script is pushed by its own dedicated piece of code, separate from the general wording-push used by every other agent, because the general one would accidentally delete the renewal agent's voicemail-message setup. That dedicated pusher was written to talk to one specific building's agent — it doesn't yet know how to say "push this to building A, not building B." When the team split the automatic pipeline by property on Aug 23 so Willows could be worked on safely, this pusher couldn't be safely folded into that automatic path, so it was deliberately left as something a person runs by hand. The team wrote this down, in the code itself, as "paused, not silent" — the intent was never to hide it, just that nobody has come back to finish the fix that would let it run safely on its own again.
The 6-hourly automatic check that watches for exactly this kind of mismatch worked perfectly — it caught this morning's gap within two minutes of the merge and has stayed loudly red ever since. The gap is real, but it hasn't gone unnoticed; it's gone un-fixed, because fixing it currently means someone manually running the one-off push button, and nobody has since Aug 15.
We went through every setting a live ElevenLabs phone agent has and checked three things: is the setting written down in the repo at all, does anything automatically push repo changes onto the live agent, and does anything automatically notice if the live agent drifts from the repo. Ranked by how much it would matter if it silently went wrong on a real call.
| Setting (what it controls) | Written in code? | Auto-pushed? | Auto-watched? | Risk if it silently drifts |
|---|---|---|---|---|
| Every setting, for Camellia specifically, since Aug 23 | Yes | No (frozen) | Partial (vs. a frozen snapshot, not the live repo) | Highest — a real, reviewed fix can sit unshipped for weeks with only the wording-specific check able to catch it |
| The literal voice/model callers hear | No (one narrow exception) | No | No | High — a wrong voice, accent, or model swap on the dashboard would go completely unnoticed |
| Whether a transfer requires the receiving person to accept the call before connecting | No | No | No | High — affects every transfer-capable agent, dashboard-only, zero record of the current setting |
| How an existing transfer behaves (e.g. conference-in vs. blind transfer) once it exists | Yes | Yes | Only partly — the check confirms the transfer exists, not that its behavior matches | Medium — a live-only edit to how a transfer works would pass the check silently |
| Webhooks that fetch a caller's name/unit before Clara answers | No | No (set once, by hand, long ago) | No | Medium-high — if one breaks after a domain/credential rotation, calls could quietly lose tenant context with no alarm |
| Which tools an agent can call | Yes | Yes (the pagination bug lived here — fixed Aug 27) | Yes | Medium — mechanism is fixed now; the cleanup from this week is what's still pending |
| Voicemail message text, for 3 specific agents (emergency relay, escalation callback, vendor outbound) | Yes, but frozen at creation time | No — no ongoing push | No | Medium — set once and never revisited; the config for one of these literally says "not auto-synced" in its own file |
| Voicemail message, renewal pair specifically | Yes | Yes (manual button) | Yes | Low-medium — same pause as the wording issue above |
| Spanish / language-detection settings | Yes, for 5 of 10 agents | Yes, for those 5 | Yes, for those 5 | Low today, but unmonitored if it's ever turned on for one of the other 5 via the dashboard |
| Which phone number rings which agent | Yes | Yes | Yes, on every pull request | Lowest — the one surface with complete, working coverage; worth copying the pattern from |
require_acceptance), for every transfer-capable agent. Currently invisible everywhere — no check anywhere would notice if this got flipped on the dashboard. It's also, separately, a setting ElevenLabs won't currently let us turn on for our account at all (see section 9) — but that's a platform gate, not a reason to skip building the watcher.sync-phone-numbers.ts:85, elevenlabs-config.ts:879, create-robo-agent.ts:126, robo-vendor/el-agent.ts:174).Separate track, same fleet: over the last two days we ran controlled test calls — Willows only, our own numbers, nothing near Camellia or a real tenant — to find out what actually happens when Clara transfers a caller to the office and nobody human picks up.
"Call screening on transfers requires feature access. Please contact support." Confirmed twice, with nothing else changed between tries — this needs ElevenLabs support to flip on for us; it isn't something we can fix from our side.Evidence: conversation conv_2301m12hjsa5fm1sa6wpkgfnyk71 (current "Conference" behavior) and conv_0901m14p5amneepr7xph05hdsvyj (new "Blind" test, Aug 28); full write-up in data/elevenlabs-snapshots/noanswer-experiments-2026-08-27.md.
A separate pass went through every voice agent and phone number in the ElevenLabs workspace — not to find drift, but to find leftover clutter nobody's cleaned up. Read-only; nothing below has been changed or deleted. Full per-agent table (evidence appendix).
Keep (24): agents holding a real phone line, wired by name into the live call-routing code, or an active bench the team is using right now — the real property phone tree, the outbound renewal and vendor-calling agents (both named directly in production settings), the owner-utilities line, the automated eval caller, and the full 8-agent Willows test fleet built for this month's transfer experiments.
Safe to delete (11): no phone number, nothing in the code or its history pointing at them, several literally named by whoever made them as temporary. Every one of these was either never called at all or hasn't been touched in a month or more.
| Agent | Why it's safe |
|---|---|
| Leasing A/B DIET (temp) | Named "safe to delete" by its own creator; zero code references, zero calls ever. |
| Verified-caller-ID proof-of-concept ("delete me") | One-off test; no code references remain. |
| SIP/DTMF touch-tone proof-of-concept | Companion test to the one above; no code references remain. |
| Vendor-QA bench — invoice scenario | One scripted test scenario; no code hook remains. |
| Vendor-QA bench — approval scenario | Same bench session, different scenario; no code hook remains. |
| Model-migration test — Leasing (temp) | Dated one-time test in its own name; never called, never referenced. |
| Model-migration test — Triage (temp) | Paired with the one above; same story. |
| Prompt-config trial, version c (temp) | Dated, versioned trial; never called, never referenced. |
| Prompt-config trial, version d (temp) | Next version of the same trial; same story. |
| "Pretend tenant" test caller | Old test script; three months quiet, no code hook remains. |
| "test agent" (explicitly marked deprecated) | The only code mention left is a historical comment, not a live link. |
Needs a look first (5): two agents were created today, apparently for an in-progress comparison test — flagging these so nobody deletes another session's live work by mistake. A 5-day-old scratch clone may still be in active use. Two older agents were the precursor experiment to the Willows fleet before it shipped Aug 23 — one had a call as recently as 6 days ago, so worth confirming that comparison is actually finished before retiring them.
8 phone numbers have no agent answering them — each still costs money every month regardless. Two of those aren't even in our Twilio account anymore, though ElevenLabs still lists them as ours. Separately, 2 Twilio numbers aren't wired to ElevenLabs at all — one plays Twilio's generic demo greeting and looks abandoned, the other is a texting-only line working as intended.
This is deleting extra copies of things Clara already has working versions of — like clearing out duplicate contacts from a phone. The fix has already been tested twice and printed the exact same answer both times.
Right now, when Clara transfers someone and nobody answers, the caller sits in silence for about two minutes before the phone company itself cuts them off with an error message — and whoever eventually looks at the office phone can't call the resident back, because the caller ID coming in is Clara's number, not theirs. Blind transfer fixes both: real caller ID, and a clean hang-up in under a minute instead of two minutes of dead air.
Of the 40 voice agents in the workspace, 11 are test leftovers nobody's using — some are even named "temp" or "delete me" by whoever made them. Cleaning those up is like clearing dead phone numbers out of a shared contacts list. The phone numbers are a separate question because releasing one is permanent and it costs money every month to keep an unused one around.
Everything below was checked twice — once by the original research pass, once by an independent verification pass reading the same source material fresh. One claim came back corrected; it's marked below. Anything not independently re-verified is labeled as such rather than presented as settled fact.
| Claim | Status | Note |
|---|---|---|
| Duplicate tools were caused by 3 automatic merge-triggered runs against Camellia, Aug 22–23, not by any manual run | Verified | Confirmed against the actual run history and the exact commit hashes involved, independently, twice. |
| The mechanism: an unpaginated tool list caused the reuse-check to miss tools past the 100th and mint duplicates | Verified | Confirmed by reading the pre-fix and post-fix code directly, both times. |
| The nightly checker had the identical pagination bug, so it was unreliable for the same window it was supposed to be watching | Verified | Confirmed by direct code inspection. |
| No live call was ever actually broken by this week's duplicate tools | Inferred | Strong supporting evidence (a documented false alarm, and past incidents of the same class always logged as "hygiene, not outage") but this is an absence of evidence, not a positive proof. Flagged, not settled. |
| Voice identity fields (voice, model, delivery) have zero code coverage anywhere in the fleet | Corrected | False as originally stated. A narrow exception exists: a Spanish-voice swap for two agents (Triage, Leasing) is tracked and checked. The base/English voice identity for every agent, and the Spanish voice for every other agent, is genuinely untracked. |
| 92 tools are safe to delete; 37 duplicate name-groups; the specific per-agent before/after counts | Verified | Pulled directly from the actual dry-run output this morning; matches the alert numbers exactly, character for character. |
| The renewal push was deliberately paused on Aug 23, documented in the code as "paused, not silent," with a real fix from this morning going undelivered as a result | Verified | Confirmed against the actual code comment, the actual merge and check timestamps (2 minutes apart), and the manual-run history (last run Aug 15, none since). |
| The renewal-specific manual re-enable path has literally never been run | Verified | Checked directly against the run history: zero runs, ever. |
| A meaningful share of the 92 flagged tools are leftovers from an unrelated internal AI-model comparison, already flagged and held pending approval since Aug 24 | Verified | Cross-checked the earlier internal note against this morning's dry-run output line by line — the same named leftover agents show zero change, confirming they're still just sitting there untouched. |
| The gap matrix in section 7 (webhooks, transfer accept-first, transfer-behavior fields) — coverage state of each | Verified | Each row confirmed by direct inspection of the relevant code, independently, twice. |
| Willows auto-updates on every merge; Camellia only via a manual, magic-word-confirmed job not run in 30 days; vendor-outreach and emergency-relay agents have no update path at all; phone-number routing is the only lane that also self-heals overnight | Verified | Confirmed directly against the workflow files and 30 days of run history. |
The wait-for-someone-to-accept transfer feature (require_acceptance) is blocked for our ElevenLabs account with an explicit "feature access" error | Verified | Reproduced twice via isolated API calls, identical error both times; config confirmed unchanged after each attempt. |
| "Blind" transfer passes through the caller's real phone number and ends a no-answer call in roughly 55 seconds, versus ~2 minutes of dead air then a carrier error on the current setup | Verified | Confirmed from Twilio's own call records and ElevenLabs' conversation transcript/timing data for both transfer types, on a test call. |
| Workspace inventory — 40 agents total, split 24 keep / 11 safe-to-delete / 5 needs-a-look, plus 8 unassigned phone numbers | Verified | Cross-checked against phone-line assignments, each agent's actual call history, and a full codebase search for every agent ID. |
Agent-workspace cleanup inventory: folded into section 10 above — full per-agent detail lives in the linked evidence appendix.