1. TL;DR (updated 2026-09-29, after the Willows test)
Recommendation: hold production on today's model (v3). Keep the Willows test property on v4 Turbo.
- ElevenLabs shipped v4 Turbo on 2026-09-28. Same price as today's model. The speech engine starts faster (about 90 ms vs 157 ms).
- But callers do not feel it. On 30 real recordings at Willows the caller waits about 2.1 s for Clara's reply on v4 Turbo, against about 1.7 s on today's model in production. Voice quality is fine: no garbling, no dead air.
- Interruption is unproven. On Fede's own Willows call he talked over a 24-second answer and Clara neither stopped nor heard him. The robot cannot talk over Clara, so this needs a timed-audio test caller (a Twilio call from a tester number; needs Fede's go).
- Shipped and live: the speech model is now one global repo setting for all of PropFlow (never per customer), pinned to v3 everywhere except the Willows test fleet, which runs v4 Turbo. Switching everyone is a one-line change.
- Found along the way at Willows, unrelated to the speech model: an 11 to 17 s first leasing answer, a "let me pick up where we left off" filler at hand-off, a wrong first name in the greeting, and tour-booking tool failures. These need their own fixes.
- Also on this page: the 30-day guest card audit (section 5) and the expression-tags experiment (section 6).
2. What v4 Turbo is
- Released 2026-09-28. ElevenLabs claims about 150 ms median time to first speech and 90+ languages.
- Both
eleven_v4andeleven_v4_turboare available on our account. - Production Clara runs
eleven_v3_conversational(voice Belle, stability 0.5, streaming latency 3, language model claude-sonnet-4-6).
Sources: TechCrunch launch coverage · ElevenLabs model docs
3. How we tested
Show the method
- Production baseline: ElevenLabs' own per-turn timings on the last 30 real triage calls and 10 real leasing calls.
- Arm A and Arm B: throwaway copies of production triage and leasing (current prompts, production tools and webhooks, triage-to-leasing handoff pointed at the copied leasing agent). Arm A stays on v3. Arm B uses v4 Turbo.
- Volume: 3 robot calls per script, 2 scripts per arm. Same robot caller voice (Mark, eleven_turbo_v2). Same time window (01:23–01:45 UTC, 2026-09-29).
- Metrics (ElevenLabs per-turn timings): language model time to first token, language model first sentence, voice engine time to first byte, and audio since silence (how long the caller waits after they stop talking).
- Judge: the same Fable judge the nightly review uses.
4. Results
Full results table
All times in milliseconds. “n turns” is the number of agent turns measured.
| Metric (ms) | p50 | p90 | mean | n turns |
|---|---|---|---|---|
| Production baseline, v3 (real calls) — triage | ||||
| LLM time to first token | 902 | 1424 | 1017 | 142 |
| LLM first sentence | 1062 | 1760 | 1272 | – |
| Voice-engine time to first byte | 155 | 197 | 158 | – |
| Audio since silence | 1871 | 3173 | 2862 | 123 |
| Production baseline, v3 (real calls) — leasing | ||||
| LLM time to first token | 942 | 1469 | 1086 | 45 |
| LLM first sentence | 1260 | 1768 | 1364 | – |
| Voice-engine time to first byte | 280 | 441 | 320 | – |
| Audio since silence | 2299 | 4493 | 2552 | 18 |
| Arm A, exact clone on v3 — triage | ||||
| LLM time to first token | 957 | 1552 | 1072 | 23 |
| LLM first sentence | 1405 | 2265 | 1523 | – |
| Voice-engine time to first byte | 143 | 161 | 144 | 26 |
| Audio since silence | 1774 | 3389 | 1858 | 23 |
| Arm A, exact clone on v3 — leasing | ||||
| LLM time to first token | 989 | 1358 | 1030 | 23 |
| LLM first sentence | 1324 | 1706 | 1367 | – |
| Voice-engine time to first byte | 134 | 154 | 135 | 26 |
| Audio since silence | 1781 | 2412 | 1756 | 26 |
| Arm A, exact clone on v3 — overall | ||||
| Voice-engine time to first byte | 138 | 161 | 139 | 52 |
| Audio since silence | 1774 | 2894 | 1804 | 49 |
| Arm B, same clone on v4 Turbo — triage | ||||
| LLM time to first token | 916 | 1617 | 1066 | 17 |
| LLM first sentence | 1179 | 1942 | 1346 | – |
| Voice-engine time to first byte | 82 | 91 | 83 | 20 |
| Audio since silence | 2224 | 3549 | 2322 | 17 |
| Arm B, same clone on v4 Turbo — leasing | ||||
| LLM time to first token | 941 | 1100 | 943 | 21 |
| LLM first sentence | 1288 | 1422 | 1268 | – |
| Voice-engine time to first byte | 83 | 91 | 83 | 24 |
| Audio since silence | 3152 | 3694 | 2749 | 24 |
| Arm B, same clone on v4 Turbo — overall | ||||
| Voice-engine time to first byte | 83 | 91 | 83 | 44 |
| Audio since silence | 2890 | 3709 | 2572 | 41 |
Judge pass / fail
| Script | Arm A (v3) | Arm B (v4 Turbo) |
|---|---|---|
| Triage | 0 of 3 pass | 0 of 3 pass |
| Leasing | 2 of 3 pass | 1 of 3 pass |
The triage script is a known-behavior scenario that fails on today's production prompt too. It is not a model effect.
What the numbers mean
The voice engine is 55 ms faster per turn on v4 Turbo (139 to 83 ms mean). The language model takes about 950 ms on both, and it dominates what a caller hears, so the felt change is small. Audio since silence is higher in the v4 arm, which we did not expect.
Why audio-since-silence went up
We broke every v4 turn down into its parts: how long Clara waited after the caller stopped, the language model’s time to its first sentence, the voice engine’s time to its first byte, and whatever is left over.
- Not the wait after the caller stops. Median 0.29 s on v4, and only the first reply after the greeting is long.
- Not tool calls. The leasing calls made none; the triage calls made one hand-off each, 19–28 ms.
- Not the voice engine’s first byte. That fell to 83 ms, as advertised.
- The leftover. After subtracting all of the above, v4 Turbo still has a median 1.55 s (leasing 1.6 s, triage 1.0 s) before the caller hears audio. On v3 the same leftover was about 0.3 s. That is the whole regression.
The leftover comes in two sizes: about 0.4–1.0 s on turns that open with a full sentence, and about 1.5–1.9 s on turns that open with a short acknowledgement (“Great!”, “Perfect!”, “Got it”) followed by a question. Leasing is hit harder because its qualifying turns are mostly that shape. Working hypothesis, unproven: v4 Turbo reports its first byte on an early packet, then takes over a second to start real speech on a short fragment. Two ways to confirm: measure leading silence in the response audio, or merge the acknowledgement into the next sentence on one call. Transcripts showed no speech artifacts (no stage directions read aloud, no cut-off words); every call ended by the caller hanging up, none dropped.
5. Guest card audit, last 30 days (2026-08-29 to 2026-09-28)
A read-only count of AppFolio guest cards created in the window, for the companies that have data in production. Test and staff companies are excluded.
Takeaways
- 68% of cards are system-created from listing sites.
- Hand-typed cards come from “Assistant PM” (a shared login, 13), Kat Barker (11), Erika Wieser (3) and Clara herself (4).
- Situs and Yale 25 show zero because nothing mirrors their cards yet.
Guest card tables and caveats
Volume per company
| Company | Guest cards created (AppFolio) | Inquiry rows in window (cross-check) |
|---|---|---|
| Western Slope | 109 | 161 |
| Camellia / JP&Co | 24 | 41 |
| Situs Group | 0 (no rows) | 0 |
| Yale 25 Station | 0 (no rows) | 0 |
| Total | 133 | 202 |
The inquiry count is higher because it includes every prospect row on our side, not only AppFolio cards. Situs Group has no properties in production. Yale 25 Station has no mirrored cards.
Lead source (as AppFolio records it, 133 cards)
| Lead source | Count | % |
|---|---|---|
| Zillow Rental Network | 41 | 30.8% |
| Apartments.com | 20 | 15.0% |
| No Source Recorded | 13 | 9.8% |
| Website | 12 | 9.0% |
| Rent. (Rent.com) | 12 | 9.0% |
| Zillow | 12 | 9.0% |
| Referral | 5 | 3.8% |
| Caller Didn't Provide Source | 4 | 3.0% |
| Phone Call | 4 | 3.0% |
| Realtor.com | 3 | 2.3% |
| Uloop | 2 | 1.5% |
| Zumper | 2 | 1.5% |
| Shared Application Link | 2 | 1.5% |
| Rentler | 1 | 0.8% |
| Walk-in | 0 | 0% |
Zillow combined: 53 (39.8%). No source at all (“No Source Recorded” plus “Caller Didn't Provide Source”): 17 (12.8%).
Who created them (robot read of all 133 cards)
| Company | Cards | System | Staff by name | Staff, blank name | Unknown | Named people / labels |
|---|---|---|---|---|---|---|
| Western Slope | 109 | 73 (67%) | 28 (26%) | 6 (6%) | 2 (2%) | Assistant PM 13, Kat Barker 11, Clara Propflow 4 |
| Camellia / JP&Co | 24 | 17 (71%) | 3 (12%) | 4 (17%) | 0 (0%) | Erika Wieser 3 |
| All | 133 | 90 (68%) | 31 (23%) | 10 (8%) | 2 (2%) | Assistant PM 13, Kat Barker 11, Clara Propflow 4, Erika Wieser 3 |
“Assistant PM” is a shared login label, not a person. Creator comes from AppFolio's own activity log (Guest Card Received = system, Guest Card Created = staff). 133 cards read, 0 errors.
Lead source by creator
| Lead source | Cards | System | Staff (named or blank) | Unknown |
|---|---|---|---|---|
| Zillow Rental Network | 41 | 38 | 3 | 0 |
| Apartments.com | 20 | 20 | 0 | 0 |
| No Source Recorded | 13 | 4 | 9 | 0 |
| Rent. | 12 | 12 | 0 | 0 |
| Zillow | 12 | 1 | 11 | 0 |
| Website | 12 | 7 | 5 | 0 |
| Referral | 5 | 0 | 5 | 0 |
| Caller Didn't Provide Source | 4 | 0 | 2 | 2 |
| Phone Call | 4 | 0 | 4 | 0 |
| Realtor.com | 3 | 3 | 0 | 0 |
| Uloop | 2 | 2 | 0 | 0 |
| Shared Application Link | 2 | 0 | 2 | 0 |
| Zumper | 2 | 2 | 0 | 0 |
| Rentler | 1 | 1 | 0 | 0 |
Per week
| Week | Western Slope | Camellia | Total |
|---|---|---|---|
| Aug 29 – Sep 4 | 25 | 9 | 34 |
| Sep 5 – Sep 11 | 28 | 2 | 30 |
| Sep 12 – Sep 18 | 29 | 4 | 33 |
| Sep 19 – Sep 28 (10 days) | 27 | 9 | 36 |
| Total | 109 | 24 | 133 |
Caveats
- Before the robot read, only 34 of 133 cards had a creator on record. The robot read covers all 133.
- Our own earlier creator stamp was unreliable: seven cards were stamped “staff” with a note line instead of a person. The robot read shows the real creator (Erika Wieser on three, blank on four).
- Camellia's mirror looks thin and may be incomplete, and started only on 2026-09-20. Treat 24 as a floor; 41 is the upper bound from our inquiry rows, which include non-AppFolio rows.
- Western Slope's mirror looks complete (1,065 guest cards mirrored against a 1,172-card export; the gap is unexplained). The 109 is the number we trust most.
- Two counts, not one reconciled number: the AppFolio mirror (133) versus our inquiry rows (202).
- Dates use AppFolio's “Received” time, not the time we synced the card. The last week is 10 days long.
- Lead source is AppFolio's wording. “Zillow” and “Zillow Rental Network” are kept separate in the table and combined in the note. We did not verify what “Website” means.
- Yale 25 Station showing zero is a real gap only if its cards should be pulled; we did not check the sync setup.
- “Assigned user” in AppFolio is not the creator and was not used.
6. What else v4 brings, and the rollout plan
Facts below come from ElevenLabs' own API and docs unless marked otherwise. ElevenLabs publishes no migration guide from v3 to v4 and no end-of-life date for v3 Conversational; the only staged-migration advice found is third-party.
What is new in v4 / v4 Turbo
| Feature | What it means for Clara | Status |
|---|---|---|
| Speed: v4 Turbo first byte ~83 ms per turn on our bench (v3: 138 ms) | Smaller share of the pause a caller hears; the language model is ~0.95 s of it | Measured 2026-09-29 |
| Cost: v4 Turbo is 0.5x characters, same multiplier as v3 Conversational (ElevenLabs /v1/models) | No change to the bill for the swap | Verified |
| Languages: 85 in the API list (marketed as 90+), v3: 74 | Spanish handling unchanged; more languages available later | Not needed yet |
| Expression: inline audio tags such as [pause] or [reassuring] are honored by v4 | Our prompts strip tags today because v3 Conversational rendered them as dead air; v4 could re-open warmer delivery | Test at Willows before any prompt uses tags |
| Voice adherence: stability and similarity controls unchanged, stronger accent hold (third-party report) | Keep Belle's settings as they are; listen for accent drift on Spanish turns | Listen-test |
| Agents platform (September changelog): parallel tool execution on by default; "Procedures" task instruction sets; new realtime speech-recognition models (scribe v2 realtime turbo/lite) with secondary languages; call queueing at concurrency limit; Slack alerting | Parallel tools and the new recognizer are worth a bench trial; Procedures overlap with our specialist prompts | Backlog, named owner needed |
Expression tags experiment (arm C, 2026-09-29)
Fede asked us to try v4's inline direction tags in experimentation. Arm C = the same two clones on v4 Turbo with expressive mode on and four tags offered (reassuring, warm, apologetic, pause), six robocalls, same scripts as arm B.
| Metric (p50 / p90, ms) | Arm B: v4 Turbo, expression off | Arm C: v4 Turbo, expression on |
|---|---|---|
| LLM first byte | 934 / 1266 | 922 / 1391 |
| TTS first byte | 83 / 91 | 86 / 124 |
| Audio since silence | 2890 / 3709 | 3025 / 3695 |
| Judge passes | 1 of 6 | 2 of 6 |
Dead air did not come back. Every call's audio was re-transcribed and scanned for pauses of 1.2 s or more inside Clara's turns: none. No tag word was spoken aloud. That was the failure that retired expression on v3 in August, and v4 Turbo did not reproduce it.
But the model never used a tag. No agent line in any of the six calls carried a bracket cue, so there is no evidence yet that the tags change how Clara sounds. Six calls is also too few to read the judge difference as real. Next: a larger batch with a prompt nudge that invites the tags, and Fede's listen on the bound line. Until then expression stays off in the repo; the four-tag change is uncommitted in an experiment worktree only.
Runner changes shipped in PR 13529 (an --apply-sound flag and per-call transcript saving, which closes the gap where deleted clones lost their conversations).
Rollout plan, three steps
| Step | What | Who decides |
|---|---|---|
| 1 | Make the speech model a repo setting per fleet, pinned to today's v3 value; merging changed nothing (PR 13506, merged 2026-09-29) | Session 006, merges under the standing grant |
| 2 | Flip the Willows staging fleet to v4 Turbo through the normal sync (PR 13612, merged 2026-09-29 08:24Z; live check the same day: both Willows clones on eleven_v4_turbo, production Triage and Leasing still on eleven_v3_conversational); prove with fastlane runs on both scripts, the nightly judge for a few days, and Fede's listen; bar = same judge scores, no new failure class, and the 1.5 s short-turn gap not present on real calls | Session 006 builds; Fede listens |
| 3 | Production fleets move one at a time, only on Fede's go | Fede |
Fede's standing goal for every voice change: more natural, faster, and at least as precise as today.
7. Willows proof on the real test line (2026-09-29)
Verdict: hold production. On real recordings, callers wait about 0.4 s longer for Clara on v4 Turbo than on today's model. The speech engine itself is faster, but the caller does not feel it. Interruption could not be proven either way.
What ran: 30 graded robot calls into the Willows line (v4 Turbo), 128 Clara turns, pause measured from the call audio (end of the caller's last word to Clara's first word). Comparison: 24 real production calls on today's model (68 turns). The comparison is indicative, not like for like: production callers are people, Willows callers were the robot.
| Measure | v4 Turbo, Willows | Today's model (v3), production |
|---|---|---|
| Pause the caller hears, median / slow tail (ms) | 2130 / 3540 | 1710 / 3004 |
| Speech engine first byte, median (ms) | 91 | 157 |
| Language model first byte, median (ms) | 878 | 938 |
| Silences of 1.2 s or more inside a spoken turn | 0 of 128 | 0 of 68 |
| Garbled or cut-off speech | none found | n/a |
| Vendor's own "audio since silence" timer, median (ms) | 3140 | 1802 |
- The overnight "1.5 s gap" was mostly a measurement effect. The vendor timer overstates the real pause by about 0.8 s on v4 Turbo and 0.2 s on v3. The audible difference is about 0.4 s, and it is the same on short and long replies.
- Interruption: not proven. On Fede's own call (14:28Z) he spoke over Clara's 24-second first leasing answer; his words are audible in the recording but Clara neither stopped nor transcribed them. Settings match production except the speech model. The robot cannot talk over Clara, so the ten talk-over calls produced no overlap. A timed-audio test caller is needed to settle it.
- Judge: 2 of 30 passed, but the failures are about what Clara says and does, not how she sounds, and the triage script also fails on v3. Found along the way at Willows, independent of the speech model: the first leasing answer runs 11 to 17 s; a filler line "let me pick up where we left off" plays at hand-off (13 times here, never in production); the greeting used the wrong first name; the booking tool failed or was walked back in 9 calls.
| Bar | Status |
|---|---|
| Pause at or below today's model | Not met (2130 vs 1710 ms median) |
| No new failure type | Met on speech quality; interruption untested |
| Judge score at or above today's model | Cannot confirm (too few v3 robot calls) |
Files: ~/agents/006/v4-turbo-trial-2026-09-28/willows-proof/ (report, 40 call recordings and transcripts, analysis scripts). Runner: PR 13746.
8. Decisions for Fede
- Production speech model. Recommendation: hold on today's model. v4 Turbo adds about 0.4 s to every reply as measured today and interruption is unproven. Revisit when either changes. Yes / hold?
- Interruption test. Recommendation: go. A test caller that plays recorded speech at a set moment, placed through Twilio from a tester number, 10 calls on Willows and 10 on the bench line. About 60 lines plus two recorded clips; it is the only way to answer the talk-over question. Go / hold?
- Willows. Recommendation: keep it on v4 Turbo so testing continues and the nightly judge keeps scoring it. Yes / hold?
- Willows flow defects (long first answer, hand-off filler, wrong name, booking tool). Recommendation: fix as ordinary bugs in their own small PRs, independent of this trial. Yes / hold?
The production switch, when it comes, is a one-line change to the global setting, shipped through the normal merge-to-main sync, and reverted the same way.
9. Artifacts
- Test runner: PR 13472
- Run ids:
2026-09-29T01-23-46-731Z(Arm A, v3) and2026-09-29T01-37-44-289Z(Arm B, v4 Turbo)