Ask Clara · Overnight build report — 2026-08-01

Ask Clara went from timing out on the founder's own question to answering all seven — and now starts answering in a few seconds

Everything below was measured on production by the same graded seven-question owner battery, run before and after. The whole rebuild is merged and live. Judge: Claude Sonnet 4.6, five dimensions (correctness, breadth, specialization, precision, actionability), 1–5.

1 · The trigger

In Friday's standup Sean asked Clara for the top 5 problems I have as an owner — and it timed out in front of everyone. The investigation that followed found the failure was not flaky, it was deterministic: that question returned a platform timeout 3 out of 3 times at roughly 60 seconds, and so did every other broad, portfolio-wide owner question. The same question, hinted to use fewer lookups, came back fine in 42 seconds — so the intelligence was never the problem, the budget was.

Two quieter failures were sitting underneath it. Some questions returned a successful response with a completely empty body — the screen showed a blank Clara bubble, no error, no log line. Others were cut off mid-word and shipped as if finished, so the action items at the end of the answer — the whole point — were silently lost. And nothing streamed: the entire progressive-typing apparatus in the interface was inert scaffolding waiting on a server that sent one buffered chunk at the very end.

Eight verified root causes were catalogued in total, from the hard 60-second wall clock and the unchecked answer cap down to a poisoned chat history that stuck in one browser forever once an empty answer was saved into it.

2 · The before scoreboard

Seven owner questions, run against production the night of Jul 31.

4
hard failures (timeouts)
2
empty answers
5
truncated answers
1
of 7 gradable — scored 4.4
61s
median total time
≈ total
time to first word

Time-to-first-word was identical to total time on every single question: you stared at nothing for a minute, then either got everything at once or got an error.

3 · What was rebuilt overnight

Making an answer possible at all

Making it feel like a product

Making her smarter — the brain layer

Ask Clara's rebuilt chat interface running on production
The rebuilt interface, live on production.
Ask Clara composer with the voice dictation control
Voice dictation in the composer.

4 · The after scoreboard

Same battery, same judge, same production URL. The streaming release merged during the session, so the headline numbers below are from the final battery run against live production with everything shipped.

Live on production, everything shipped

7 / 7
answered
0
empty answers
0
truncated answers
4.66
overall quality (1–5)
2.3–6.7s
time to first word, every question
4.1s
median time to first word

Sean's question — "top 5 problems I have as an owner" — now starts answering in 2.3 seconds and grades 4.8 out of 5. Before, it was a 60-second wait ending in an error, three times out of three.

An honest note on total time: median end-to-end is about 64 seconds, in the same range as before. What changed is not how long Clara thinks — it is that you now see the first words in 2–7 seconds and watch her narrate which part of the portfolio she's reading while she works. The wait didn't shrink; it stopped being a blank screen.

Question by question

Question Before Prod, wave 1 Live grade Live first word
Portfolio health timed out 5.0 (rerun) 4.85.5s
Top 5 owner problems Sean's question timed out 4.8 4.82.3s
Where to pay attention timed out 5.0 5.04.0s
Rent per square foot empty answer 4.6 4.46.7s
Vendor and maintenance themes answered but truncated · 4.4 4.2 4.82.9s
Delinquency empty answer 4.6 4.65.1s
Drill-down follow-up timed out 4.4 4.24.1s
Overall 1 of 7 gradable · 4.4 7 of 7 · 4.6 4.664.1s median

Breadth — how many parts of the portfolio one answer reaches across — was the dimension that stayed soft all night, and it is the one the brain layer moved most: 3.7 on the wave-1 run, 4.4 live. Specialization and actionability are now a clean 5.0 across all seven questions.

Supporting evidence — the streaming release's preview

Before it merged, the streaming release was benchmarked on its own live preview: 7/7 answered, 0 empty, 0 truncated, overall 4.46, first word in 1.3–7.4s (median 3.2s), median total 43s. That run is what justified shipping it; the production numbers above supersede it.

Two independent checks

5 · Also fixed, from the same standup

6 · Where it landed, and what's still open

All twelve pull requests from this session are merged, and every Trello card that came out of the standup is Done. The whole rebuild — time budget, answer budget, error handling, retry, boot, dictation, interface, streaming, persona and brain layer — is on production.

Deliberately left open:

7 · 2026-08-02: Model decision — Sonnet 5 at low effort

The Opus-vs-Sonnet comparison deferred above was run the next morning on an in-process harness (subscription-judged, same seven-question owner battery), followed by an effort sweep and a fresh live production re-measurement.

Tier comparison, against the shipped prompt

Metric Opus 5 Sonnet 5
Overall4.664.49
Correctness4.574.57
Specialization5.004.86
Precision4.574.14

Correctness tied exactly. The overall gap is 0.17, mostly precision (4.57 vs 4.14). Latency between runs was noisier than the quality gap: Sonnet measured 0.76× Opus in one run and 1.16× in another.

The pre-agreed quality gate — overall ≥ 4.5 — technically failed by 0.01 against the shipped prompt. Fede chose Sonnet with the numbers in hand, for cost and latency. It is recorded in code as an explicit override, not a measured win.

Effort sweep on Sonnet 5 — low vs medium

Two repeats per leg.

Metric Low Medium
Overall4.364.30
Correctness4.364.14
p50 latency29.8s46.2s
Slowest question59s136s

Correctness was higher at low effort in both repeats; quality overall is labeled unchanged within variance. Latency is not: p50 0.64× and the slowest question 0.43×. CHAT_EFFORT is set to low.

Latency changes shipped the same morning

Live production re-measurement — same seven-question battery, 2026-08-02

7 / 7
answered
0
empty or truncated
4.2
overall quality (1–5)
30s
median total (was 64s — 0.47×)
63s
slowest (was 107s)
3.1–8.5s
time to first word

Plainly: the 4.2 vs 4.66 quality dip is a single run and sits within the run-to-run variance already observed. The rig re-runs in about 10 minutes if answers feel thinner in real use.

A precision-focused prompt variant was also tested and rejected — it raised precision by +0.29 but cost breadth on both models.

PropFlow Docs