What is shipped, what is building and what waits on Fede for the Clara coworker rewrite.
Status is DERIVED from each row’s own PR, decision and prod-evidence links by
bin/refresh-tracker clara-new-hire-phase-tracker — never typed. Last refreshed
2026-11-03 01:42Z.
Closing: critical-only plan, Nov 2 Rows below carry their step (Step 1) and read in that order. Nov 2: Fede ordered the initiative closed on a critical-only plan; Closing plan DM'd to Fede Nov 2 06:45Z (both max inspections agree): C1 every rewrite, retry-regenerate and redraft layer deleted (Fede, Nov 1 night: the "one retry-once for dash / no-tour-times" exception is gone; checks hold into Supervised review only; misfiring checks get their wiring and context fixed), C2 held review for texts and emails plus two check fixes, C3 DONE Nov 2 (bench with Fede's Wrong cards frozen, plus the prospect_reply surface), C4 runbook section DONE (live on the master page; the pointer PR is open), C5 Situs autonomous watch. About 63 non-critical rows close, cancel or hand off in the morning after Fede reads it. Decider: Fede.
A rewrite that still fires on Sonnet (dashes: 57 of 71 first emails) waits for step 5.
Goal (Fede, Oct 27): "inspect, consolidate, eliminate tech debt, to clear the way to a more autonomous opus 5.5 with less checks but only if safe." Order follows Fede's Oct 26 ruling: refactor on the same Sonnet first, then the new model. Dependency order, never calendar: a box starts when the arrow into it has its proof.
Today: one drafting loop writes every text and email (first email, prospect reply, tenant text). One prompt builder renders the shared parts (property info, knowledge, language, voice) plus at most one job body per turn: leasing 2,373 lines, maintenance 470, turnover 217. The model is picked per channel, not per module: email and text on Sonnet 5, video and staff desk on Sonnet 5.5. Follow-up texts come from a separate writer on Haiku 4.5; card notes on Opus 5. No per-module or per-customer model setting exists, only one global override. Western Slope leasing is live and autonomous; Situs first emails are held for review; tenant texts are off everywhere.
| Step | Needs first | What changes for a customer | Who flips |
|---|---|---|---|
| 0. Baseline | Nothing | Nothing | None |
| 1. Code-only debt | 0 | Nothing | None |
| 2. No prompt split needed (modules are already separate bodies) | Nothing | Nothing | None |
| 3. Writer model named per module in code | 0 | Nothing (same values as today) | None |
| 4. New model in shadow per module, leasing first (Sonnet 5.5 and Opus 5.5 arms) | 1 and 3 | Nothing (shadow) | None |
| 5. New model live for one module, everywhere, leasing first | 4 | The new model writes that module's replies at every customer | Fede |
| 6. Next module (maintenance = tenant texts, supervised first), then turnover | 5 | The new model on each module in turn | Fede, each |
Every row opens: when it was added, why it exists, and the links that prove it. Rows waiting on a human carry a Decide → link straight to the answerable question on the decisions page. Statuses are re-derived from live PR, decision and prod-evidence state by bin/refresh-trackers; nothing here is typed by hand. Merged and shipped both count as done (Gera, 2026-10-19); a prod_evidence permalink is what moves a row from Merged to Shipped.
The Willows is our test property; every text, email and call below was a test persona on our own lines (prospect tester +17207348022, bench tenant +17202122112, Willows line +18442853526). Other numbers are cut to their last 4 digits. Each verdict is copied from that run’s report; “Not exercised” means the check never actually ran. The bench tenant’s thread is wiped nightly, so this table is the durable record of it.
| Scenario | Lane | Verdict | Ids | Issue / note |
|---|---|---|---|---|
| Voucher / Section 8 question, English: plain yes, same screening for everyone | Prospect text | PASS | msg_2decb875, SM2ae3109f | #22681 (fix) |
| Voucher question, Spanish: same yes, in Spanish | Prospect text | PASS | msg_3815f980 | #22681 (fix) |
| Card note: one short line for the whole voucher exchange, read back from the AppFolio card | Prospect text | PASS | card 311, note written 11:16Z | #22676 (fix) |
| 'How many bedrooms is it?': names the one home in play correctly (201, two-bed), no roster | Prospect text | PASS | msg_05da2c0f | - |
| 'Can you send me the application link?': no promise made, pointed to the Oct 22 email | Prospect text | Not exercised | msg_046417da | - |
| 'I can't find it, send it again': Clara said she re-sent it; nothing was sent and no hand-off | Prospect text | FAIL | msg_63bf147e | #22777, fix #22794 merged; live re-run not exercised yet (deploy pending) |
| 'How many units are open?': count held back, draft and check name kept on the stop records | Prospect text | PASS | dstop_2a316a25, dstop_ef1fb194 | - (renter got no reply) |
| Follow-up text due at night waits until 8 AM Denver | Prospect text | PASS | prospect-outreach-6c91245d | - |
| Follow-up schedule intact on cards 351-353 | Prospect text | PARTIAL | prospect-outreach-351289bc / 2299df14 / 77b07b09 | #22779 (skip reason not logged) |
| Follow-up writer states true bedroom sizes (cards 351-353) | Prospect text | Not exercised | dstop_5187d255, dstop_b1c0f29f, dstop_b90c17e8 | #22966 |
| No dashes: a draft with a dash was caught and re-drafted | Prospect text | PASS | dstop_77528b19 | - |
| Each repeat text is its own row, none merged | Prospect text | PASS (texts only) | msg_d64c027a, msg_8e12e06e, msg_e6db618c, msg_d602b09f | AppFolio inquiry path not exercised |
| Cancel today's 11 AM tour: text, tour row, calendar event, reminder all gone | Prospect text | PASS | tour 2ff2c982, msg_4a0181d; no reminder text sent (16:12Z check) | - |
| 'What's available?': should name one home with its rent and special | Prospect text | FAIL | msg_1c26bc2 | #22803, fix #22827 still open |
| 'Can I see it tomorrow at 2?': booked, calendar event, one-line card note | Prospect text | PASS (booking) | tour e8c5d824 | booked on a one-bed after two-bed talk (#22804); no confirmation email (#22102); no AppFolio showing (#21393) |
| Move the tour to 4 PM: same tour moved, one text, event moved | Prospect text | PASS | tour e8c5d824, msg_1f2a031 | - |
| 'Is there parking?': answered from the knowledge base in the same turn | Prospect text | PASS | msg_af61442 | - |
| Question not in the knowledge base (water heater): team asked, but renter told to ask at the tour | Prospect text | PARTIAL | msg_dee0e28, team note msg_9ddad51 | #22807 |
| Cancel the Wednesday tour | Prospect text | PASS | tour e8c5d824, msg_6724053 | - |
| Spanish 'what's available?': Spanish and formal, but rent range instead of one home | Prospect text | PARTIAL | SM9d95e4e4 | #22803 |
| Spanish booking Thursday 10 AM: booked and confirmed in Spanish | Prospect text | PASS (booking) | tour c35c7d61, msg_587f2cf | asked for a two-bed, booked on one-bed 105 (#22804) |
| Spanish cancel of the Thursday tour | Prospect text | PASS | tour c35c7d61, msg_7a162b0 | - |
| Card notes on AppFolio card 311 for every booking, move and cancel | Prospect text | PARTIAL | card 311 | 4 notes right; one repeats the false 'resent it' claim (#22777) |
| After the fix: 'two bedroom?' then 'tomorrow at 2' should land on 201 | Prospect text | FAIL | tour 68596363 | #22804 closed, but the merged fix #22815 did not hold live |
| Door 4 facts refactor: one home's rent passes, a full unit list is held | Prospect text | PASS | msg_fff3fa3c, dstop_e7669866, dstop_7d1c53bd | #22649 (refactor) |
| New email lead, first reply (stress-domain sender) | Prospect email | Not exercised | conv 52e4f1f2 | test-building recipient guard refused the send (by design) |
| New email lead on our own test inbox | Prospect email | PARTIAL | conv 52e4f1f2, msg_370cfd3 | #22808 (filed into another lead's conversation) |
| Email 'Tomorrow at 11 AM' was held as a maintenance reply | Prospect email | FAIL | eschold_f7ed0c31 | #22805 |
| After the fix: the same email is labelled non-maintenance and delivered | Prospect email | PASS | msg_b1199763 | #22797 (fix) |
| Guest-card first email reads the calendar in its own turn (+ forced-retry count) | Prospect email | Not exercised | EMAIL#685754bb | no live sender reaches the first-email engine at Willows; #22913, #22932 |
| Vague caller hangs up at the name question | Voice | Not exercised | conv_voice_5b338603 | name never asked; call was transferred |
| Caller asks for a person: transferred to the test ring list, no staff name read out | Voice | PASS | conv_voice_73fa543d | - |
| Prospect books Thursday 3 PM by phone | Voice | FAIL | tour 738c1cc4, conv_voice_6b20ca1c | #22786, #22787 |
| Tenant: heater not working | Voice | FAIL | WILL-003990, conv_voice_0260d4c8 | #22788 |
| Spanish caller asks for a two-bedroom: answered in English all call | Voice | FAIL | conv_voice_191e023b | #22785, #22787 |
| After the fix: Spanish caller answered in Spanish every turn | Voice | PASS | EL conv_6901m5z90kzte139ehatjrtvvpa8 | #22813 (fix); still a range, no single home (#22803) |
| After the fix: phone tour books in about 4 seconds, no failure message | Voice | PASS | tour 42fc5e08 | #22802 (fix) |
| Name corrected mid-call moves the tour to the new name | Voice | Not exercised | EL conv_2901m5zabg4ffmfbt5fnkd56z3eg | #22809 merged but not on the website yet, and Clara never re-booked; English caller greeted in Spanish (no issue yet) |
| Repair text (dripping faucet): work order filed, reply held for review | Tenant text | PASS | WILL-003987 | - |
| Second, different problem gets its own work order | Tenant text | PASS | WILL-003988 | - |
| Spanish no-heat text: Spanish draft, same tone, emergency order and on-call chain | Tenant text | PASS | WILL-003989 | - |
| AppFolio shows three separate work orders | Tenant text | PASS | AppFolio 3404, 3405, 3406 | - |
| Office-hours question once repairs are open | Tenant text | FAIL | eschold_e47cdb2a | #22780, fix #22797 merged; tenant re-run not exercised |
| 'I'd like to renew' once repairs are open | Tenant text | FAIL | eschold_cc0237c2 | #22780, fix #22797 merged; tenant re-run not exercised |
| Ceiling-leak text: order filed once, but office got an urgent page about the old no-heat text | Tenant text | FAIL | WILL-003991, eschold_f9317fc9 | #22939 |
| After the fixes: no second urgent page, filed texts marked as filed, no re-tagging | Tenant text | PASS | eschold_d02f5d6e, eschold_930828fd | #22950, #22955 (fixes); new bedroom-heat report was filed onto the open no-heat order; held replies re-answered old asks |
| Slack rating buttons: one record per rated text, repeat press is not a second record | Back office | PASS | gv_64b9005a, gv_8556a330, gv_7b846d0c | no Willows card exists for a human to press |
| Slack grading lands as a grade record | Back office | PASS (buttons) | same records | reactions and thread replies do not grade (not built) |
| 15-minute no-reply alert ignores a held first email | Back office | Not exercised | - | #22776 (cannot run at Willows) |
| Re-run script refuses a live customer property, runs at Willows | Back office | PASS | card 311 | - |
| Staff welcome text: one text to a new test leasing agent, then setting off again | Back office | PASS | SMc2d85de4, staff thread 4d8415a1 | - |
Reference material for the Clara coworker rewrite: ledgers, bench results, the model inventory, both final-pass reviews and the chapter list. The design itself is on the master page: /a/clara-coworker-loop-v3-2026-10.
His order holds and overrides the earlier "one module a day" cut. Day 0 freezes the baseline, because no refactor can be judged without it. Then the base is refactored first: one core engine, the four shared interfaces, no module with its own hand-off or its own escalation; the shouldSpeak chain (PR 2) and askTheTeam/askState are that work. His three priorities sit on top of that core, in his order: (a) Situs Group onboarding done well is Stage 1 work (the intro texts, proven at the Willows on Oct 25 from the building line; the ownership map; the hello per role; the feedback loop) and does not wait for the refactor, so it runs beside it from day 1, with the turn-on at Situs being his word; (b) the hand-off framework is habit B on the shared askTheTeam and askState, built from customer feedback (the hand-off audit: one in three hand-offs that needed a person never reached one; the Situs and Western Slope rows); (c) the Opus 5.5 upgrade is the Monday model plumbing on the frozen baseline, and tenant replies (maintenance first, since intake and the attach guard are live at Western Slope; renewals second, it already has the loop) ship dark behind the existing tenant-door setting and wait for the floors and his turn-on. The full-context hypothesis is benched on the frozen set before any prompt is rebuilt around it. Leasing and collections deep-dives leave this window. Proposed, pending his review.
Frozen sets, three arms, a judge with planted controls, one row per run on the Clara Bench page. First result, 261 repair episodes:
| Measure | Production today | Same prompt, Opus | Opus + your fixes |
|---|---|---|---|
| Filed when the office filed | 45% | 48% | 57% |
| "Check with the team" | 34% | 39% | 8% |
| Invented facts | 30% | 21% | 16% |
| Duplicate of an open work order | 5% | 5% | 11% |
| Judge score (of 5) | 2.98 | 3.23 | 4.01 |
Next on the bench: restrictive-today vs full-context (top); frozen baseline v1 (Day 0); the new first-email writer in shadow (release plan); the repair path on Opus 5.5 (Thursday); Benchmark E, who is writing (not started).
One model setting per path, byte-identical replay, then one path at a time: replies and the first email (moving), voice (about 61 calls a week on Sonnet 4.6, timed call test first), the repair chain, the Haiku 4.5 checks.
Cheap, this week: the staff-card rule, three bugs, thumbs as the judging surface. Not careful at all: delete prompt text no turn needs (~62k tokens per first reply; identical replay proves it). Careful: first-reply proof and go-live (a fix, no switch). Very careful: when to reply; dark, Fede turns it on per customer.
| Item | Numbers | Status |
|---|---|---|
| Reading the lease | Review #19: lease says water included through 6/30/27; Clara said "nothing on file". Plan: a short lease sheet (about 400 words) in context, full text by tool. | Plan only ("Just a plan and architecture, don't build it.") |
| Tool coverage, 2,591 tenant texts | 34% need no reply; of 1,704, every tool makes 45% answerable. Documents alone 10%, lease alone 1%. | Read-only |
| Commercial work orders | Situs Group: 145 of 4,000 (Chase Tower 76, Wells Fargo Building 67, 6900 Lowell 2). A commercial block by property type. | Research only |
| Bench | Result |
|---|---|
| D3: office-conversation rule, reply or silent | 22 of 23 of Fede's graded examples (in-sample); 0 replies on staff-run threads, 0 invented facts; routine replies 12 of 50 |
| G2: 261 repair episodes, two outcomes | Invented a fact in 1 of 261 (was 13); filed 8 of 42 where the office filed, all right |
| Triage: 1,994 real texts | 45% need no reply; of the rest 23% Clara today, 44% need a person, 33% need more tools |
| Case (Situs Group) | Report | Clara (3b, replay) | Office | Fede's call |
|---|---|---|---|---|
| 1 · Beate, Lampliter | New stove after roaches came out of it | New stove order; note on roach order | Separate stove order | ____ |
| 2 · Cornisha, Lowry | Washer and dryer failing | On the AC order (wrong attach) | No order | ____ |
| 3 · Marly, Woodside | Kitchen extractor; roach and fly fumigation | Fumigation right; extractor on the faucet order (wrong) | Pest schedule; no hood order | ____ |
| 4 · Karin, Lampliter | AC leaking inside | On the outside spigot order (wrong) | New AC order | ____ |
| PR #22380 merged | One register for every tenant; register eval 9 of 18 failing to 18 of 18 passing. Fix, no switch. |
| PR #22383 merged | Attach guard: a report about a different thing files its own order; 51 of 51 real notes unchanged on replay. |
| PR #22373 merged | Manual send tells the opt-out gate which company it speaks for. |
| #22381 open | A two-issue text can lose one issue (1 of 8 runs). |
| #22389 open | Note-tool refusal: carry scheduling and access onto the new order, dedupe it. |
| #22385 open | Register eval follow-ups. |
"Heard today" is what I took from your call with Gera; it is my reading, not confirmed.
Update, Oct 25 evening (Benchmark lane): Day 0 is done; the frozen baseline v1 and its numbers are recorded on the Day 0 roadmap row. Nothing in it needs a decision from Fede.
| # | Question | Options | Recommendation | Heard today |
|---|---|---|---|---|
| 1 | Prompt shape | (a) one shared block + one job block per turn type, tools fixed per role · (b) a full prompt per role × channel · (c) today's builder with sections switched off | (a) | lean prompts per job |
| 2 | Where customer rules live | (a) company settings, one block · (b) a file per customer · (c) the shared body | (a) | |
| 3 | Model for the first-reply writer | (a) keep Sonnet 5 · (b) Opus 5.5 · (c) one setting for all | (b) Opus 5.5; Sonnet 5.5 medium is the close, cheaper second | one strong frontline model, Sonnet 5.5 at minimum, aiming Opus 5.5 |
| 4 | The bar for the first-reply proof | yes / change a line | yes | replays with database state are the quality bar |
| 5 | A first email fails its check, and the retry fails too | (a) hold for a person · (b) short code-built reply + team item · (c) hold, no retry | (a) retry once, then hold, as a safety net behind the new writer; a reviewer prefers (b) | |
| 6 | Which small model checks retire first | by replay evidence; fair-housing check last | start with checks the proof shows never fire | small classifier stacks are not the strategy; "no Haikus" is an order of removal, one per replay |
Also heard: you own the baseline cleanup (prompt diet, no-reply groundwork, model upgrade, replay bench); Gera keeps the maintenance flow and gatekeepers.
The release plan for any new writer on a live path, starting with the first email to prospects, with its two questions:
Template for every path that moves to the split prompt and Opus 5.5, starting with the first email to prospects. A new writer is a new capability on a live path, so it ships behind a company setting (default: today's writer) that is deleted the week the rollout ends. Fixes found along the way still ship with no switch.
| Stage | What happens | Customer sees | Exit when | Rollback |
|---|---|---|---|---|
| 0. Tonight | Blocking check for a first email that reads as a note about the prospect merges (a fix, no switch). New writer wired in shadow: on every real first email the new draft is written and stored beside what was sent; nothing is sent from it. | Nothing changes | Shadow running at Western Slope, Situs and Camellia | None needed |
| 1. Sun to Mon: shadow | Every real first email gets both versions. The judge scores each pair nightly. A review page shows them side by side. Fede grades his dozen plus any pair the judge flags. | Nothing changes | 50+ real pairs; no pair where the new email is worse on a fact, a hold or tone; Fede's grades at parity or better | None needed |
| 2. Mon or Tue: one company | Company setting "first-email writer: today / new", default today, visible on the company page. Fede flips one company. | New first emails at that company | 3 days, every sent email judged the same day, zero regressions Fede or the judge flag | Setting back to today, instant |
| 3. Supervised week | Same company. Daily page: sent, held, judged worse. Alert if the hold rate or judged-worse rate passes production's. | Same | 7 clean days | Same |
| 4. Everyone | Remaining companies flipped. Setting deleted the same week. | New first emails everywhere | Setting deleted | Revert PR |
Decisions for Fede: (1) Which company goes first in stage 2? Recommendation: Western Slope (highest volume, staff already review Clara's emails daily). (2) Shadow at all three companies or Western Slope only? Recommendation: all three; shadow sends nothing.
Dark: company.firstEmailWriter (today / new) | feature: first email to prospects written by the split prompt on Opus 5.5 | proof: pending (shadow pairs, stage 1) | turn-on: Fede.
Fifteen real first emails, today's Clara and the new one side by side; Fede's marks decide whether the new one goes live:
What to do: 15 real renter inquiries. Each one shows today's Clara and new Clara, written on the same facts in the same hour. Mark each one today, new or same. Your marks decide whether new Clara goes live. Nothing is live yet.
How this was run. The new prompt was frozen and run 3 times, back to back with today's prompt in the same hour (about 65 cards per run, plus a Spanish slice of 4). Cards whose inputs changed between the two writers are left out. The pairs below come from the last run. Names are kept. Phones and emails are removed. A red dash marks where the email used an em or en dash.
| Measure | Today's Clara | New Clara | Better |
|---|---|---|---|
| Emails with a dash in them | 55 / 55 / 54 | 0 / 0 / 0 | new |
| Gave the home's beds, baths, size and rent | 5 / 7 / 6 | 29 / 29 / 30 | new |
| Offered real tour times when the calendar had them | 51/60 / 51/59 / 46/56 | 58/60 / 56/59 / 55/56 | new |
| Kept the tour window ("between 10:00 AM and 2:30 PM") instead of one clock time | 31 / 24 / 23 | 50 / 50 / 47 | new |
| Left something the renter raised unanswered (judge, best of 3) | 7 / 10 / 11 | 3 / 4 / 1 | new |
| Wrote the email before looking at the calendar | 14 / 14 / 16 | 3 / 3 / 1 | new |
| Put an answer off ("I'll check with the team", "at the tour") | 3 / 3 / 4 | 6 / 5 / 6 | today |
| Named extra homes or said how many are open (the count check fired) | 5 / 5 / 5 | 5 / 6 / 3 | same |
| Held by a send check (bench clock mix-ups left out, see the note) | 10 / 6 / 10 | 4 / 5 / 6 | new |
| Read like a note about the renter, not an email to them | 0 / 0 / 0 | 0 / 2 / 0 | today (both now held) |
| Typical length, words | 53.0 / 51 / 50.5 | 67.5 / 68 / 69.0 | today (shorter) |
| Typical time to write, seconds | 40.2 / 38.4 / 42.9 | 25.2 / 23.8 / 24.7 | new |
| Spanish (4 cards): opened with "Hola" | 2 / 4 / 3 | 4 / 4 / 4 | new |
| Spanish: emails with a dash | 2 / 2 / 2 | 0 / 0 / 0 | new |
| Spanish: how natural it reads, 1 to 5 (judge) | 2.6 / 4.5 / 4.2 | 4.3 / 4.8 / 4.5 | new |
Pairs compared per run: 66 / 67 / 64.
| From | Danielle Coleman via Zillow |
|---|---|
| Property | Camellia Apartments |
| Message | I would like to schedule a tour.. Pet Description: A pet. Pet Type: Other. Lease Length Months: 12. |
What differs: New gives the pet fees only, adds the move-in special that fits her 12-month lease, and offers a real tour window. Today recites the pet approval steps, asks what kind of pet, and offers no time.
| From | Steven Fernandez via Zumper |
|---|---|
| Property | Northridge Apartments |
| Message | Info Request: Move-in 10/09/2026 I saw your rental listing on Zumper and would like to learn more. Zumper Listing: [link removed] |
What differs: Close. New keeps the full weekend windows where today picks two clock times. New still leaves out the rent and size.
| From | Anelia Hazelwood via Zillow |
|---|---|
| Property | 2841 Morningside Court |
| Message | I'm interested in your property. Do you accept section 8 Housing participants? |
What differs: Today puts the Section 8 question off to the tour. New says yes, with the same screening as anyone, and gives the home's details.
| From | Natalee Madsen via Zillow |
|---|---|
| Property | Ember Estates |
| Message | Hello, I'd like more information about NOW PRE-LEASING ! 1.5 Months FREE ! Brand-New Duplex Homes ! Private Garages & Yards.. |
What differs: Today ignores the 1.5 months free and the garages they quoted. New says plainly the special is not on file, but invents a private yard from a smoking rule.
| From | Byron |
|---|---|
| Property | Barcelona Apartments |
| Message | Si asepto condisiones y espero respuesta |
What differs: Both are in Spanish. New adds the address and keeps the tour windows. Today offers two clock times.
| From | Angely |
|---|---|
| Property | Lampliter |
| Message | Estoy interesada por favor mandarme para aplicar lo más pronto posible gracias |
What differs: Both say the application link is not on file and the team will follow up. New has no dash and keeps the windows.
| From | Angela |
|---|---|
| Property | Barcelona Apartments |
| Message | Listing Notes - Me interesa un apartamento de 2 cuartos Hablo español |
What differs: Today opens "Hi Angela" in a Spanish email and uses dashes. New opens "Hola Angela" and adds her move-in date.
| From | Queen Olafadehan via Zillow |
|---|---|
| Property | Camellia Apartments |
| Message | Hello, I'm interested in a studio apartment at Camellia Apartments (unit 306/606) and am looking to move in around October 14. Do you offer 6-month lease options? I'd also like to know what additional monthly fees are charged besides the base rent, such as utilities, parking, trash, or other community fees. Thank you!. Pet Description: no pets. Pet Type: None. Lease Length Months: 6. |
What differs: Both answer the 6-month lease and the fees in full. New gives the studio's rent and size and says the special is for 12-month leases only. Both are long.
| From | Nathan Law via Apartments.com |
|---|---|
| Property | 2978 Emerald Circle |
| Message | Hello I’m interested in the house however have a few questions. First is there other pictures of the inside of the house or back yard. Also is there any security deposit or any other fees to rent the property prior to moving in.. Reason For Moving: None. |
What differs: Both list three other homes, against your one-home rule. New also gives each one's full details. Both were held by a send check.
| From | Tanya Christians via Zillow |
|---|---|
| Property | 2841 Morningside Court |
| Message | I would like to schedule a tour. Possibly Friday morning or Saturday? |
What differs: Today has no greeting and asks for her name. New greets her, gives the home's details, and offers Saturday plus next Friday morning. New was held only by the bench clock mix-up.
| From | Amanda Derryberry via Apartments.com |
|---|---|
| Property | 2841 Morningside Court |
| Message | Do you have a viewing available around 4?. Reason For Moving: None. |
What differs: Today has no greeting and offers 2:30 on Saturday or Sunday. New greets her, gives details, and offers Saturday 2:30 or Monday at 4.
| From | Patricia Li via Zillow |
|---|---|
| Property | Camellia Apartments |
| Message | Dear owner Leasing Agent. |
What differs: Today names the open studio and offers times. New says it will check with the team on which studios are open, when the facts show one open. That is a step back.
| From | Yoira Estrada via Zillow |
|---|---|
| Property | Camellia Apartments |
| Message | Hello, I'd like more information about Camellia Apartments.. |
What differs: New offers only Saturday where today offers both days. It also adds the deposit, which nobody asked about. Both mention the 1-bedrooms too.
| From | Jaleigh Monsen via Zillow |
|---|---|
| Property | 2841 Morningside Court |
| Message | Hi! Would a 6-9 month lease be considered for this? |
What differs: Both pass the 6 to 9 month question to the team. New first says the standard lease is 12 months and that some homes allow 6.
| From | Keyvis Gilmore via Zillow |
|---|---|
| Property | 2863 Florida Street |
| Message | I'm interested in your property and would like to move forward. Can you send me an application for this property? Message in Zillow Rental Manager: [link removed] Pet Description: no pets. Pet Type: None. Lease Length Months: 12. |
What differs: Both say the link is not on file. New adds the home's details, the $50 fee and the lease term, and keeps the windows.
From the frozen runs on Oct 24, 9:11 PM to 11:20 PM MT. The production change for the new first email stays on a local branch, with no test runs, until you mark these.
The four repair cases in full, with a blank for Fede's call on each (summary in tenant communications):
Written Oct 25 from the repair benchmark. Two of these (cases 2 and 3) are wrong attaches in the accepted prompt itself, not duplicates: the report went onto an unrelated open order because the matching order was not in the list Clara saw.
Four real Situs Group tenant texts. Each time an open work order was nearby, and the question is whether Clara should add the report to it or file a new one. "Clara" below is the accepted version (3b, the production prompt split into pieces, running on Opus 5.5). It is a replay only; nothing was sent. Times are Mountain time.
Two things to know first:
Tenant text (Fri Aug 28 2026, 5:12 PM, replying to the office's pest-prep message):
Thank you for sending these
I will get this list completed
I would really like a new stove soon too please. I can't think about using that after what I saw crawling out of it.
Open order the harness saw:
What the office did: On Aug 31 it created a separate stove order, LAMP-18184, "Stove roach infestation: Please inspect the stove, as the tenant reports it is heavily infested with roaches and is requesting a replacement." It asked for permission to enter. Beate said yes and added: "I had added this to the roach submission on the portal but maybe I should have made it a separate one." The office replied: "I have created a separate work order."
What Clara (3b) did: She filed a new order for the stove and added a note to the roach order about the prep list.
Ugh, I'm sorry you saw that, Beate. No wonder you don't want to use the stove. Your stove request is in as work order LAMP-18717, but replacing it needs the team's approval, so I can't promise that part here, and there's no scheduled time on it yet. I've also added a note to LAMP-18152 that you're working through the prep list.
(Our labeler counted this as a duplicate of the roach order. The office made the same choice Clara did.)
Question: Is a new-stove request after roaches came out of the stove the same order as the roach treatment, or a new order?
Fede's call: ______________________
Tenant text (Sun Jul 12 2026, 11:17 AM, answering the office's Jul 10 question about the dryer):
Good morning I believe there's a problem with the washer and the dryer. The washer does not retain enough water to wash the clothes. They don't really be wet after the cycle is done then when you put them to the dryer by time, the cycle of the dryer is done. The clothes are already mild dude. I believe they need a replacement.
("mild dude" is most likely "mildewed".)
Open order the harness saw:
In the tenant's history, status unknown, not in the harness's open list:
What the office did: No work order in our copy. The next office message (Jul 13) was about an outstanding balance.
What Clara (3b) did: She attached the washer and dryer report to the AC order (a wrong attach).
Good morning Cornisha, ugh, mildewy laundry is the worst. I've added the details to work order LOWR-17614: the washer isn't filling with enough water, and the clothes come out of the dryer smelling mildewy. Replacing the machines isn't something I can approve here, but the tech will see your note when they look at both machines.
Question: If the May dryer order were still open, would this report go on it, or is a washer and dryer that do not work a new order? (Either way it never belongs on the AC order.)
Fede's call: ______________________
Tenant text (Wed Jul 22 2026, 12:57 PM):
Buenas tardes , para preguntar sobre una solicitud que hice hace días para la revisión de mi extractor de olores de la cocina y para una fumigación de cucarachas y moscas
English: Good afternoon, I'm asking about a request I made a few days ago to check my kitchen odor extractor and for a cockroach and fly fumigation.
Open orders the harness saw:
In the tenant's history, status unknown, not in the harness's open list:
What the office did: Replied: "Your unit has been added to next week's pest control schedule and will be treated during the scheduled service." It filed no order for the kitchen hood. The next day it filed an unrelated garbage disposal order (WOOD-17738).
What Clara (3b) did: She put the fumigation on the roach order, which is right, and the kitchen extractor on the bathroom faucet order, which is a wrong attach.
Buenas tardes, Marly. La fumigación de cucarachas está en la orden WOOD-17721, que está abierta y asignada a Situs Real Estate Corp. Ya le agregué que también necesita tratamiento para las moscas.
La revisión del extractor de la cocina está en la orden WOOD-17489, que también sigue abierta. Por ahora ninguna de las dos tiene una fecha programada.
English: Good afternoon, Marly. The cockroach fumigation is on order WOOD-17721, which is open and assigned to Situs Real Estate Corp. I added that it also needs treatment for the flies. The kitchen extractor check is on order WOOD-17489, which is also still open. Neither one has a scheduled date yet.
Question: The kitchen hood is not on any order Clara can see as open. Should Clara file a new order for it, or treat it as the old September hood order?
Fede's call: ______________________
Context: The day before, office teammate Deiby Magana texted: "Buenas tardes Karin, si te arreglaron tu aire? Me dijo mantenimiento que esta semana van y te lo remplazan como sea con uno mas nuevo." (English: Good afternoon Karin, did they fix your AC? Maintenance told me that this week they'll go and replace it with a newer one either way.)
Tenant text (Sun Jul 26 2026, 11:42 AM):
Hola buenas tardes si fíjate que si pero fíjate que se sale agua mucha agua porque tengo que poner trapos
English: Hi, good afternoon, yes it was, but a lot of water is coming out and I have to put down rags.
Open order the harness saw:
What the office did: Two days later (Jul 28) it filed a new order, LAMP-17787, "AC unit water leak: The tenant reports that the AC unit keeps leaking water."
What Clara (3b) did: She added the AC leak to the spigot order (a wrong attach) and reported it to the team.
Hola Karin, qué pena que el aire esté soltando tanta agua. Ya lo agregué a su orden LAMP-17337 y le avisé al equipo de la propiedad que es urgente. Mientras tanto, si puede, apague el aire desde el termostato para que deje de salir agua.
English: Hi Karin, sorry the AC is letting out so much water. I added it to your order LAMP-17337 and told the property team it's urgent. In the meantime, if you can, turn off the AC at the thermostat so the water stops.
Question: An AC leaking inside and an outside spigot leak: same order or new order? (The office filed new.)
Fede's call: ______________________
Two adversaries read the sources and the draft, not the conclusions: a Fable 5.1 reviewer (two rounds: the design, then a gap hunt) and an Opus 5.5 reviewer. Each item below shows both positions and the choice. Items marked "recorded for Fede" are where we kept his ask against the reviewers' advice, or left a conflict for him; they are recorded here, not asked (decisions for Fede holds three questions).
| Topic | Fable 5.1 said | Opus 5.5 said | What we chose |
|---|---|---|---|
| The spine | Four stages 'the same for every layer' blur a one-time setup with a per-layer loop; one failure lands in three stages. | The page said most cases were lost before the hand-off while its own prototype put 8 of 12 at the hand-off; the placement rule is partly circular and the prototype numbered stages differently. | Setup once, loop per layer, habits cross-cutting (the lifecycle); the contradicted sentence is gone; the prototype footer matches the page. adopted |
| Learning and asking are one mechanism | Batch asks at learning and single asks live are the same record. | The ask record is the existing escalation matter (owner, waiting-on-staff, answer); the 17 ways Clara reaches staff need a fold / keep / delete table, and the closing rule contradicted itself. | 'The ask is the unit' (learn and get feedback); the record is the matter, tied to a person, with end states; the 17-row table is on the page (Figure 13); picked-up vs answered split (decided here and the hand-off layers figure (dropped Oct 25)). adopted |
| Supervised as a hold queue | 'A person releases each draft' rebuilds the queue Fede called the problem; shadow-and-diff needs no clicks; 'Fede today' is the manual-QA role he refused. | Same finding, with the record: 95 of 142 and 35 of 35 expired; a live call cannot wait for a release; about 25 releases a day at one customer (the reviewer's estimate). Round 2: the queue still survived on Clara's own channels, with no end. | Shadow where a person already answers; a hold only where the customer's own system has the button (Send in AppFolio). Round 2 removed the release queue on Clara's own channels: she answers covered topics live, asks for the rest, and is corrected afterwards (supervised). partly adopted, then adopted after round 2 |
| Where the hand-off state lives | The ask record is leasing-shaped; the state is the work order, the offer, the ledger step; the showcased non-question cases still fail. | Agrees the ask is the matter; adds that owners today are addresses, not people, and most staff answers happen where we cannot see them. | State on the owning record, ask only when none exists; owners are people; the invisible-answer problem is why shadow reads the PMS mirror (supervised, the five habits, decided here). adopted |
| Readiness | Coverage plus released-unchanged measures knowledge and a click; the judge is lenient; PropFlow sets the floor. | 'Released unchanged' cannot be computed (zero releases at Situs); the brain has no reader; add owner coverage with a measured reply rate. Round 2: hand-off correctness had no floor, and 'agreement' could only mean matching today's Clara. | Three numbers plus the owner precondition, re-measured continuously with a way back down (learn and get feedback); round 2 added a floor on all three, replaced percent match with the clean shadow diff, and rewrote question 1 so its options differ for renters. partly, then adopted (round 2) |
| Three-cats before/after | (not raised) | The 'verbatim' turn was misquoted and the 'after' promised a follow-up nobody at Camellia would give; production had held the forward on purpose; the prototype's own later turns tell the truth instead. | Rebuilt from the real renter turns 4 and 5; the 'after' is the prototype at renter turn 5 (one numbering, the prototype's renter turns 4, 5, 6); a tracked ask only where an owner agreed to reply (the five habits). adopted |
| The replay numbers | Stage placement is rule-driven for 12 of 14; zeros are structural; 'production 45%' is a selected-failure rate. | The grader disagrees with the human audit on 13 of 33; two 'moments' are not renter hand-offs; two 'right' answers send renters to a main line that looped; the planted-defect check ran on one thread. | Headline tile removed; the hand-off layers figure (dropped Oct 25) and the prototype carry every caveat; the numbers stay as a replay, never a verdict. adopted |
| Owner calls buried in Decided | 'Asks at every company' decides against Constitution I.1; the morning list is a gated new outbound; ownership rows decide inside Gera's model. | Same three, plus: 'answers written back as policy' conflicts with Fede's Aug 20 'remember this?' rule, and the merge gate of ~979 fake scenarios conflicts with 'no fake scenarios'. | I.1 and the morning list are question 3; Gera rows are proposed to Gera; the merge gate is a 30-day real-traffic replay; saving answers as policy moves to rollout step 4; step 2 only passes the answer on and records it on the ask. adopted (round 2) Superseded Oct 24 by Fede: no “remember this?” question is asked, Clara sorts standing answers from one-offs herself. |
| Staff texting | Email-until-opt-in contradicted Fede's lean (text primary, both supported). | Same; a staff-texting setting already exists and the per-person preference row covers it. | Question 2 recommends text first for anyone who gave a cell and met Clara; the existing setting widens, dark. adopted |
| Live customers and Situs | Situs at 33% and Western Slope at 47% have no entry point. | Situs has no owner and no team email; Gera's model cannot run a hand-off with no recipient; the page never said what a Situs renter hears. | Question 1 (re-enter supervised: Clara keeps answering, every disagreement graded); the Situs sentence is in the ownership-map figure (dropped Oct 25); an owner who agreed to reply is the entry test for the habits. adopted |
| Ownership map mechanics | One company-wide table; Gera resolves nearest-wins per building. | Nearest-wins would make a building's CC list replace the company's; no fallback owner; the doer inference from Clara-routed texts is circular. | Per building; CC adds up; company fallback; doer inferred from PMS activity and confirmed (week one). adopted |
| Gaps | No senior at the customer; identity first; leaving; fair-housing floors; writes with read-back; delivered is not sent; staff give Clara work; voice; outcomes review. | Emergencies have no mode; no urgency rule or burst folding; no way back down; authority limits; ask end states; customer off-boarding (un-forward lines first); exceptions must never become policy; silence cannot authorize action. | All added: week one, learn and get feedback (way back down), 6 (habit E, exit checklist), 7 (floors incl. emergencies, urgency, authority), 16 (end states, silence rule). adopted |
| The 3-to-6-month arc | Fold the month arc into the spine. | It is an aspiration drawn as a plan; renewal turn-on slipped five times; no months in anything a customer sees. | Kept on Fede's word, labeled his aspiration everywhere it appears (the verdict, the metrics tile, Figure 1, the video caption) with today's numbers beside it; the video was re-cut the same day and no longer names a number of months. recorded for Fede |
| Autonomy axis | Levels by what Clara may do alone: talk, act, commit; per module, computed, tier as ceiling. | Per module, yes; but 'approve everything' as a base tier is the failed queue, observe is missing, and five ladders now exist (renewals 0–5, vision 0–3, playbook 0–2, rollout off/supervised/live, this page); map onto Gera's existing controls; proposes observe / live with asks / operator. | Talk, act, commit as the paid levels with observe (shadow) below them as a stage, not a tier; the other ladders are mapped in the caption of the two-levels figure (dropped Oct 25); autonomy per module is proposed to Gera. adopted Superseded the same night by Fede ("those are not the three levels"): two levels by how much the team approves before a send, from the Sheridan and Verplank scale, acting always the baseline (the two-levels figure (dropped Oct 25)). |
| Pricing on this page | One document per initiative; pricing designs the levels around price points; the switch mapping belongs on the sprawl page. | Price on scope only; autonomy is earned by readiness, never sold; billing tied to a safety control turns a demotion into a refund conversation; Gera separates paid entitlement from autonomy on purpose. | Not adopted in full: Fede asked for the tie, so his figures stay marked 'not decided here'; 'approve everything' is no longer priced (the two-levels figure (dropped Oct 25)). Both reviewers' objection is recorded; the pricing design is its own page. recorded; not one of the three questions |
| Habit D and the intro have no customer ask | Constitution I.4 unmet for both. | (not raised) | Recorded on the page; Western Slope is where they get a first ask. recorded for Fede |
| Length | 4,947 words; cut 7c, fold 7b, sources into a fold. | About 5,500 visible words against an 800 target; same cuts plus the engineering rows in Decided. | The page is ~11k visible words after round 2; his 'one artifact' ruling was about scope, not length. recorded for Fede |
| Source | What it gave this page |
|---|---|
| Product vision (Aug 2026) | "The AI does the work, and asks humans for judgment"; answers become policy, asked once; one brain across modules; PMS system of record, Clara system of work; every edge case ships to every property; sensitive requests always go to a person |
| Fede's voice notes, Oct 22–24 (quoted in Trinity relays; the Slack DM is not readable by API) | The new-hire and intern model, connectors, supervised mode, layers, the coworker habits, check before you speak, the 3-to-6-month arc and 80/20, augmented judgment, "map it to what we already have", coffee dates and per-person preferences, "ask for help" instead of "escalate", meetings to groundwork, the headcount outcome, two dials and three autonomy levels tied to pricing, who owns what, the morning list, "push back, don't agree", texting staff as primary with email supported |
| Company brain design, property brain prototype, platform vision v2, platform execution | Public vs internal memory with evidence, scope, audience, dates and speaker; an answer saves only if public (Oct 7); recap-and-correct instead of an approval queue; three walls against leaks; the ladder to full context; autonomy per lane; teaching by staff; what is merged-but-off (notes store, email intake, meeting join) and that no reader exists; connectors as a recommendation |
| Renewals team loop, engine one-pager, levels of autonomy | Learn → confirm → recipe → replay; one gate inside their system; read-before-write and read-back; autonomy as presets; "if a human starts it we don't intervene"; timers only wake, state decides; D15: no confirm screens |
| Agentic building blocks (now a pointer here) | The seven blocks and the module standing table; hand-off as layers; "sent never counts as seen"; the morning list design |
| Clara onboarding playbook, intake plan, Situs hand-off, dynamic onboarding mock, Sep 7 onboarding north star | How we learn the office today, the 80–90% bar, the 90% send-as-is stage exit, Situs's 33% coverage, the pre-fill that never saved, what onboarding does not capture (owners, preferences, approval chains, module processes) |
| Human hand-off framework (now a pointer here) and the Oct 24 RCA | The 810-row audit, the state rule and its replay, options A/B/C, the three-cats record, lead-to-lease flow with 12 of 13 steps touching a person |
| Clara staff loop v3 (this URL, Oct 16) and Clara meets the team (now a pointer here) | The 17 ways Clara reaches staff, one-desk option, the ~979-scenario harness, the intro flow, the ownership map, the Western Slope no-reply data |
| Sep 6 hand-off deck, escalation journey rethink, Camellia silent hand-off, staff notification architecture, PMS messaging hand-off | Rules already decided: one hand-off sentence, never promise a time, three outcomes only, no per-property setting, "hold mode is the problem", never touch eviction or judgment calls; 33 doors mapped, 16 of 17 one-engine PRs closed |
| Testing playbook | The ladder; fixes climb to Willows, new capabilities to the sandbox company |
| Gera's portfolio architecture | Company as the wall, nearest-wins settings, roles as scoped rows that end with the role, four job areas, no approver role; connectors attach at the company |
| Research passes, Oct 22–24 (Claude Duet and three other assistants' first-run flows; how ownership is modelled elsewhere; what offices leave unanswered; how the best AI product videos are cut) | the outside research table (dropped in the Oct 25 rewrite) and the three hello rules; the owner / approver / CC shape; the no-reply numbers behind the morning list; the video's pace, music and motion rules |
| Oct 21 Western Slope working session (raw transcript) | The customer named both maintenance staff; recap said leasing first, transcript says maintenance first; the morning list is the one habit a customer asked for |
One line per box on the master page's machine diagrams: what it does, the file or function that proves it, which modules use it. Items marked unverified were not traced end to end.
# Clara's machine: verified facts (origin/main edfd125bdc, 2026-10-25)
Read only from `/Users/federicochapa/repos/propflowai-map-ro`. Every line: box name | what it does | proof (file : function) | who uses it.
Module tags: SH shared core, LE leasing, MA maintenance, RE renewals, TC tenant comms (resident questions; no module directory of its own).
Where last night's inventory (commit 47c1188a19) disagrees with today's code, today's code wins and the delta is named in "Deltas" at the end.
"UNVERIFIED" means looked for and not proven.
## A. Inbound channels and entry points
- A1 Text message (Twilio) | STOP/START/HELP handled first, then a message envelope is put on the inbound queue; if the queue is off or the message is too big, it is processed inline | `src/app/api/twilio/webhook/route.ts` : POST, `publishInboundMessage`, fallback `processEnvelope` | SH
- A2 Voice call transcript (after the call) | Phone is out of scope: the live call turn happens inside ElevenLabs. After the call, the transcript enters here and is written to the conversation; prospect and work order are captured | `src/app/api/voice/call-ended/route.ts` (re-entered by `lambda/inbound-processor/handler.ts` : `processVoice`) | SH
- A3 Email to our own address (SES) | SES stores mail in S3 and a Lambda puts a pointer on the queue; webhook fallback exists | `lambda/ses-email-forwarder/index.mjs`, `src/app/api/email/ingest/route.ts`; handler `processEmail` → `agents/clara/lib/email/process-inbound-email.ts` : `processInboundEmail` | SH
- A4 Connected Outlook mailbox (push) | Microsoft Graph webhook, writes an email intake record, then queues `inbox_email` | `src/app/api/integrations/outlook/webhook/route.ts` → `src/lib/domain/email/inbound-delivery.ts` : `deliverInboundEmail` → `src/lib/integrations/email/webhook-processors.ts` : `processNotification` | SH
- A5 Company mailbox pollers (pull) | Every minute, same path as A4; the Gmail poller skips companies that are not live | `lambda/company-mailbox-poller/handler.ts` (`pollCompanyMailboxes`), `lambda/gmail-company-inbox-poller/handler.ts` (`pollGmailCompanyMailboxes`) | SH
- A6 New guest card from the PMS | The AppFolio sync imports guest cards into the mirror and emits `lead.received`; a cron drains it into the first-email consumer | `lambda/appfolio-sync/handler.ts` → `src/lib/domain/pms/mirror/guest-card-mirror.ts` : `emitLeadReceived`; `src/app/api/cron/lead-received-outbox-drain/route.ts`; `src/lib/domain/leads/consumers/clara-first-email-consumer.ts` | LE
- A7 Reply on a guest card / PMS inbox | `wakeClara` queues a `pms_email` payload or a text envelope with carrier `appfolio`; the PMS inbox mirror wake does the same for renter texts on Clara's PMS thread | `lambda/appfolio-sync/handler.ts` : `wakeClara`; `lambda/appfolio-sync/pms-mirror-reply-wake.ts` : `mirrorReplyWake`; consumer `agents/clara/lib/messaging/process-pms-email.ts` : `processPmsEmail` | LE (PMS email), SH (text)
- A8 Staff texts and emails | No separate door: they arrive through A1/A3/A4 and are recognised as staff by the router (rung 1b), then get staff-desk tools | `src/lib/domain/identity/resolve-actor.ts` : `resolveActor`, called from `agents/clara/lib/messaging/inbound-router.ts` : `routeInboundMessage` | SH
- A9 Staff approve or answer a held draft | Slack thread replies, the dashboard held-email route, and the email staff-reply door (tagged reply key, exact approve/reject words) | `src/app/api/slack/events/route.ts`; `src/app/api/held-emails/route.ts`; `src/lib/domain/staff-reply/door.ts` → `resolve-held.ts` | SH
- A10 Video | Separate video turn surface | `src/app/api/video-turn/chat/completions/route.ts`, `src/app/api/video-clara/*` | SH (out of scope for the diagram beyond a mention)
- A11 Dashboard "reply as Clara" | UNVERIFIED: no production route found under `src/app/api` (searched takeover / as-clara / staff-reply patterns). The operator skill `clara-takeover` is tooling, not the app.
### The spine (queue to turn)
- S1 Inbound queue | One SQS FIFO queue, grouped per sender | `src/lib/integrations/sqs/client.ts` : `publishInboundMessage` | SH
- S2 Queue consumer | Routes envelopes to `processEnvelope`; older payloads (email, voice, inbox_email, pms_email, transfer_recording) through a switch | `lambda/inbound-processor/handler.ts` : `handler` | SH
- S3 Text lane | Wraps the turn with the supervised hold hook, then dispatches | `agents/clara/lib/messaging/process-envelope.ts` : `processEnvelope` → `routeAndDispatchEnvelope` | SH
- S4 Email lanes (3) | SES, Outlook/Graph (`processInboxEmail`) and PMS email all go through one record processor and a `deliver()` callback | `agents/clara/lib/email/process-email-record.ts` : `processEmailRecord`; `lambda/inbound-processor/handler.ts` : `processInboxEmail`; `process-pms-email.ts` : `processPmsEmail` | SH
- S5 Fixed-rule lanes (before any model) | In order: safety pre-empts (test menu, test-data guard, life-safety gas/CO lexicon with a canned evacuate/911 reply and a page, no model), scheduling lane, maintenance state-first lane, rating lane, maintenance intake lane, then the default lane | `agents/clara/lib/messaging/inbound-dispatcher.ts` : `dispatchInbound`, `tryPreempts`, `tryLifeSafetyPreempt`, `trySchedulingLane`, `tryMaintenanceStateFirstLane`, `tryRatingLane`, `tryIntakeLane` | SH (lanes are MA heavy: rating, state-first, intake)
- S6 Router rungs | Vendor check, PM confirmation and mass-comms replies, tenant email capture, staff via `resolveActor`, handler pipeline, tour pipeline, PM commitment extraction, then "always pass to Clara" | `agents/clara/lib/messaging/inbound-router.ts` : `routeInboundMessage` | SH (vendor, tour = LE)
- S7 Agent turn | `handleIncomingMessage` → `handleIncomingMessageTurn` | `agents/clara/lib/agent/conversation-manager.ts` | SH
## B. Per-turn inputs
- B1 Conversation history | Thread messages filtered for agent context, converted to model messages, trimmed to fit 80,000 estimated tokens (always keeps the first message plus the newest that fit; no fixed count) | `conversation-manager.ts` : `filterMessagesForAgentContext`, `buildClaudeMessages`, `applyContextWindow`, `MAX_CONVERSATION_TOKENS = 80_000` | SH
- B2 Knowledge block (not retrieval) | Full dump of company sections then property sections, minus retired items; fees rendered separately and left out for residents | `conversation-manager.ts` : `loadTurnPropertyContext`; `src/lib/domain/leasing/knowledge/leasing-knowledge-block.ts` : `buildLeasingKnowledgeBlock`; `agents/clara/lib/agent/property-knowledge-block.ts` : `renderPropertyKnowledgeBlock` | SH (named for leasing, used on every turn)
- B3 Who is writing | Identity resolution, tenant by phone, occupancy, spine name | `src/lib/domain/identity/resolve.ts` : `resolveIdentity`; `conversation-manager.ts` : `getTenantByPhone`, `resolveTenantOccupancy`, `resolveSpineNameView` | SH
- B4 Prospect and guest card | Prospect record by person, listing card for the reply | `conversation-manager.ts` : `getProspectByPersonId`, `resolveListingCardForReply` (`listing-card-for-reply.ts`) | LE
- B5 Lease answers | Lease-term context for resident questions | `conversation-manager.ts` : `resolveLeaseAnswerContext` | TC
- B6 Renewal state | Notice days and the renewal saga's paperwork | `conversation-manager.ts` : `loadNoticeToMoveOutDays`, `paperworkCopyOfSaga`; prompt block `buildRenewalPromptBlock` | RE
- B7 Turnover context | `conversation-manager.ts` : `resolveTurnoverContext` | MA (turnover)
- B8 Company and property settings | Team facts, company voice (never-say list + language), application-link mode, video tour link, tour hours, office hours | `conversation-manager.ts` : `buildTeamFacts`; `src/lib/domain/onboarding/company-voice.ts` : `readCompanyVoice`; `getOrgApplicationLinkModeForProperty`, `getOrgVideoTourLinkForProperty`, `readOrgTourHours` | SH
- B9 Persona | Name "Clara" is hardcoded; no per-company persona name field found (looked, none) | `agents/clara/lib/agent/clara-personality.ts` : `CLARA_CORE_PERSONALITY` | SH
- B10 PMS mirror | Mirrored PMS messages reach the turn only inside history (`msg.pmsMirror`, rendered by `mirrorBody`); no separate per-turn PMS lookup found | `conversation-manager.ts` : `buildClaudeMessages` | SH
- B11 Open asks and held state | Open escalation matters and the open-matter decision; hold state from the deferred reply row | `src/lib/domain/escalation/matter-context.ts` : `loadOpenMatters`; `conversation-manager.ts` : `decideOpenMatterTurn`; `agents/clara/lib/agent/deferred-reply-row.ts` | SH
- B12 Opt-out / mute | Suppression read at the start of the turn | `conversation-manager.ts` : `checkSuppression` (`muteGate`) | SH
- B13 Turn-time context messages | Injected as `<system_context>` user messages, not system prompt: completed actions, available units, open matters, cadence events, open homes, team answer, conversation state, recap, local date/time, scheduled wake (last) | `conversation-manager.ts` : `renderAvailableUnitsBlock`, `renderOpenMatterBlock`, `buildOpenHomesContext`, `computeConversationState` / `formatStateBlockForInjection` (`conversation-state.ts`), `currentLocalTimeLine` | SH
- B14 Memory / promises store | UNVERIFIED as a turn input: no memory store is read by the turn. Promises are extracted after the turn (G7)
## C. Prompt layers, in order
System prompt from `agents/clara/lib/agent/clara-unified.ts` : `getUnifiedSystemPrompt`, one template, this order:
- C1 Persona | `clara-personality.ts` : `CLARA_CORE_PERSONALITY` | SH
- C2 Today line, denied-applicant header, company voice pointer | `todayDenver`, `buildDeniedApplicantScopeHeader`, `company-voice-block.ts` : `renderCompanyVoicePointer` | SH
- C3 No dead-end hand-off rule | `no-dead-end-handoff.ts` : `NO_DEAD_END_HANDOFF_RULE` | SH
- C4 Property info | office hours, application link, knowledge sections (B2), fee rules, listing card, guest-card first-touch rules, walk-in rule | inline in `clara-unified.ts`; `guestCardFirstTouchRules`, `getApplicationLinkInstruction` | SH + LE parts
- C5 Resident scope rail (residents only) | inline | TC
- C6 Property knowledge (fees; not for residents) | `property-knowledge-block.ts` : `renderPropertyKnowledgeBlock` | SH
- C7 Specials, lease-answer grounding, application status | `buildLeasingSpecialsBlock`, `buildLeaseAnswerGroundingBlock`, `buildApplicationStatusBlock` | LE, TC
- C8 Prospect info, stated facts, tour state | inline | LE
- C9 Tenant info | `formatTenantInfo` | TC
- C10 Renewal info | `buildRenewalPromptBlock` | RE
- C11 Module workflow (zero or one) | maintenance `clara-maintenance.ts` : `getClaraMaintenanceWorkflow`; leasing `clara-leasing.ts` : `getClaraLeasingWorkflow`; turnover `clara-turnover.ts` : `getClaraTurnoverWorkflow`; staff-only `STAFF_FAIR_HOUSING_RULE` | per module (own copy)
- C12 Channel style rules | `clara-delivery.ts` : `getDeliveryRules` | SH
- C13 Language + company voice | `renderCompanyLanguageRule`, `renderCompanyVoiceBlock` | SH
- C14 Current caller / active work order | inline | MA
- C15 Denied-applicant scope (always last) | `DENIED_APPLICANT_SCOPE` | LE
Then the turn adds (in `conversation-manager.ts`):
- C16 Promoted principles (lessons from grading) | `composeWithPromotedPrinciples`, `loadPromotedPrinciplesBlock` (`src/lib/domain/grading/injection*.ts`) | SH
- C17 Capability prompt blocks (from the tool set) | `composeWithCapabilityBlock` | per module
- C18 Email overlay (email leasing turns) | `agents/clara/lib/prompts/clara-email-core.ts` : `composeEmailCoreOverlayForReplyTurn` | LE
- C19 Staff desk persona or PM persona | `staff-desk-persona.ts` : `composeStaffDeskSystemPrompt`; `clara-pm.ts` : `applyPmPersona` | SH (staff)
- C20 Thinking-off rule, transfer rule, "why line" rule | `draft-thinking.ts` : `thinkingOffRuleFor`; `transferCapabilityRuleFor`; `safety/decision-why.ts` : `WHY_LINE_RULE` | SH
- C21 Examples | No separate examples file and no few-shot block. Examples live inline in module prompts: `clara-leasing.ts` (about 105 grep hits for example/good/bad), `clara-maintenance.ts` (about 5), renewals `renewal-core.ts` and `tools/renewal/prompt.ts`, a few in `capabilities/resident-services.ts` | per module
- C22 Guardrails | No separate guardrails layer; rules live inside the workflow bodies and C3/C15/C20 | SH
- C23 Tool definitions | `agents/clara/lib/agent/capabilities/index.ts` : `composeCapabilities` picks capabilities (tenant: maintenance + resident services, + renewal when active; prospect: leasing; staff: desk/portfolio/metrics; always escalate-PM and conversation closure), `composeTools` dedupes and strips | per module (see table)
- C24 Prompt caching | up to four 1-hour breakpoints: last tool, whole system prompt as one block, newest message; no static/dynamic split inside the system prompt | `conversation-manager.ts` : `toolsForCaching`, `systemForCaching`, `withMessageCacheBreakpoint` | SH
## D. The model call(s)
- D1 Text and email turn | `claude-sonnet-5`, high effort, 16,384 max tokens, adaptive thinking; hardcoded, not env-overridable | `src/lib/platform/ai/model-tiers.ts` : `REASON_FIRST_DRAFTING_MODEL`; `conversation-manager.ts` : `resolveDraftParams`, `REASON_FIRST_DRAFTING_CHANNELS = ['email','sms']` | SH
- D2 Staff desk text turn | `claude-sonnet-5-5` | `model-tiers.ts` : `STAFF_DESK_DRAFTING_MODEL` | SH (staff)
- D3 Video turn | `claude-sonnet-5-5`, thinking off | `model-tiers.ts` : `VIDEO_DRAFTING_MODEL` | out of scope
- D4 Voice / other channels via this loop | env-dependent: `process.env.CLAUDE_MODEL`, default `claude-sonnet-4-6` | `model-tiers.ts` : `AGENT_MODEL` | out of scope
- D5 Tool loop | the model calls tools, tool results go back, up to 10 rounds (3 on the tour-action-only path) | `conversation-manager.ts` : `modelCall` (wraps `tracedClaudeCall`), `handleToolCall` (defined `agents/clara/lib/agent/tools/index.ts`), `MAX_TOOL_LOOP_ITERATIONS = 10`, `TOUR_ACTION_ONLY_MAX_ITERATIONS = 3` | SH
- D6 Side classifiers | all `claude-haiku-4-5` (`FAST_MODEL`): reply-topic verdict for the hold (`src/lib/domain/leasing/renewal-orchestration/classify-reply-topic.ts` : `classifyReplyTopicVerdict`), escalated-topic classifier (`agents/clara/lib/agent/escalated-topic-classifier.ts`), email classify (`agents/clara/lib/email/classify.ts`) | SH
- D7 Provider | Anthropic API only (no Bedrock found); no fallback model; credential rotation on 429/401 and a secondary key on 401/403; retry + circuit breaker | `src/lib/platform/ai/client.ts` (`failoverFetch`), `src/lib/platform/resilience.ts` : `claudePolicy` | SH
## E. Post-model layers (all deterministic; no LLM judge in the live turn)
In execution order inside `conversation-manager.ts` after the model returns:
- E1 Final text and "why" extraction | `safety/decision-why` : `extractDecisionWhy`
- E2 Empty or intermediate draft rescue | `decideIntermediateDraftSalvage`, `fallBackToEarlierTurnText`
- E3 Strip leaks | `stripReasoningPreamble`, `stripHistoryStamps`, `stripToolCallMarkup`, `stripInternalPrompt` (`outbound-safety-pipeline.ts`)
- E4 Dates and tour times | `enforceWeekdayDates` (`src/lib/domain/leasing/weekday-date-redraft`), `detectOffTurnTourTimes` (redraft or blank) | LE
- E5 Claims must be grounded | `enforceWorkOrderClaimGrounding` (MA), `enforceNoUnitCount` (LE)
- E6 Fair housing and promise backing (one retry) | `runOutboundSafetyChecks`
- E7 Work order confirmation, language | `ensureWorkOrderConfirmation`, `detectLanguage`
- E8 Remove private info | `stripOutboundPII`, `resolveRedactedReply`
- E9 No dashes | `enforceNoDashes` (turn-integrity)
- E10 Made-up facts guard | `guards/hallucination-guard.ts` : `runHallucinationGuard` (can hand to a person)
- E11 Final re-check | `sendGateFinding`
- E12 Anomaly review | `anomaly-review-gate` : `runAnomalyReviewGate` (may hold the draft and send a holding message)
- E13 Desk and turnover honesty | `evaluateTurnoverEditHonesty`, `stripDeskRefEcho`, `evaluateDeskWriteClaims`
- E14 Team-inbox silence | withholds a bare hand-off ack
- E15 Hand-off promise rewrite | `decideSameTurnEscalationOutbound`, `rewriteUnreachedForwardPromise`, `stripAllHandoffPromiseClauses`
- E16 Hold hook (row line) | `deferred-reply-row.ts` : `claraReplyRowWithheld` / `claraReplyRowHoldIf`
- E17 Unbacked hand-off claim | `src/lib/domain/compliance/policies/promise-backing-policy` : `findUnsanctionedHandoffClaim`
- E18 Opt-out withhold (texts) | `opted-out-reply.ts` : `withholdsReplyAfterOptOut`
- E19 Outcome + reply row | `publishFinalTurnOutcome`, `pushTurnRow(buildMessage(...))`, `saveConversation`
- Not found: a repeated-reply dedupe (only message-id dedupe in `saveConversation`); a post-model length or segment limit.
## F. Output gates, in evaluation order
### F-text (text reply; `process-envelope.ts` : `routeAndDispatchEnvelope`)
- F1 Stay quiet | Turn returns no text with a reason: runaway ceiling, farewell loop, closer skip, escalated thread (a person took over; `escalated-gate.ts`), tapback reaction, person muted, opted out, conversation ended, team-inbox silence | `conversation-manager.ts` (`suppressedBy`)
- F2 Turn failed | rate limit or error: nothing sent, logged, rethrown | `handleRateLimitExhausted`, `handleInboundTurnFailure`
- F3 Hold for approval (row line) | handyman texts Supervised, or the reply's topic / recipient / routed module is Supervised (rules R1 to R7 below); staff never held | `deferred-reply-row.ts` holdIf → `src/lib/domain/modules/reply-hold-decision.ts` : `shouldHoldClaraReply({kind:'reply'})` → `supervised-reply-rule.ts` : `decideSupervisedReplyHold`
- F4 Hold: router-answered maintenance acks and handyman in-house divert | `process-envelope.ts`
- F5 Hold: routed backstop (residents, not urgent) | `shouldHoldClaraReply({kind:'routed'})` → `routedModuleSupervisedAt`; `markReplyRowHeld`
- F6 Write the hold + approval card | `holdReplyWithAck` → `holdDraftReplyText` (`renewal-reply-review.ts`) → `holdText` (`src/lib/domain/escalation/held-text.ts`): saves `EscalationSmsHold`, posts a card to the company's Slack channel (grouped per channel, kind, day), emails reviewers, dashboard `/admin/escalation-holds/<id>`; a resident gets one fixed acknowledgement text; a rollout release can send at once; hold write failure → `escalateUnholdableDraftReply` (Sentry + #alerts)
- F7 Policy skip | test/stress sender, thread escalated (person took over), human replied (`autoReplyHold`), texts disabled (not for replies) | `agents/clara/lib/messaging/dispatch-envelope.ts` : `applyOutboundPolicy`
- F8 Consent at dispatch | skipped for replies | `dispatch-envelope.ts` : `isAllowedToSend`
- F9 Text door compliance gate, in order | harness off switch, canary, TCPA opt-out (runs for replies), AppFolio OTP, stress, test-scope recipient, flipped-line admission, module OFF, drafts backstop (`shouldHoldClaraReply({kind:'door'})`), company not live (replies always pass), contact ceiling, write admission, then send | `src/lib/integrations/twilio/client.ts` : `runOutboundTextComplianceGate`
- F10 Send | adapter registry: Twilio text, AppFolio PMS text, (email adapter uses SendGrid) | `agents/clara/lib/messaging/dispatcher.ts` : `dispatch` → `dispatch-envelope.ts` : `dispatchEnvelope`; `adapters/twilio-sms.ts`, `adapters/appfolio-inbox-text.ts`, `adapters/appfolio-pms.ts`
- F11 After send | refused: `markReplyRowNotSent`; sent: `backstampProviderIds`, then the guest-card text note is filed to the PMS (`fileStagedGuestCardTextNotesAfterSend`)
### F-email (differences)
- F-e1 Stay quiet | same loop reasons plus ingest gates (private note, lead claimed elsewhere, lead routed to guest-card engine) and `decideEmailSendMode` = suppress (relay or aggregator sender) | `process-inbound-email.ts`
- F-e2 Hold | `supervised-reply-hold.ts` : `supervisedReplyHoldScope` / `supervisedReplyHoldFor` (topic classifier + `shouldHoldClaraReply`) and routed backstop; held via `bake-gate.ts` : `holdEscalationEmail`; the 911 reply is never held
- F-e3 Email door gate, in order | no external recipient → send; staff audience → live check only; test scope; module OFF; Supervised (`shouldHoldClaraReply({kind:'door'})`); company not live → refused and logged BLOCKED (not a held draft) | `src/lib/integrations/email/company-live-gate.ts` : `evaluateCompanyLiveGate`, via `property-graph-sender.ts` : `sendPropertyEmail`
- F-e4 Lane order differs | SES: hold before live gate. Graph lane: live gate before hold (`lambda/inbound-processor/handler.ts`). PMS email: human-replied refusal before hold (`process-pms-email.ts`)
- F-e5 Send | PMS, Graph, Gmail or SendGrid door, each with its own suppression and contact ceiling
- UNVERIFIED: a "human replied" gate at the SES or Graph send door
### The hold rule (R1 to R7, `supervised-reply-rule.ts` : `decideHold`)
- R1 Reply topic's module is Supervised → hold (urgent maintenance to non-prospect sends)
- R2 A renewal action was deferred and Renewals is Supervised → hold
- R3 Routed module is Supervised → hold (same urgent carve-out)
- R4 Renewal thread or open renewal cycle, topic "other", not staff, Renewals Supervised → hold
- R5 Prospect and Leasing Supervised → hold
- R6 No topic verdict and anything Supervised → urgent sends; known lane not Supervised sends; else hold
- R7 Otherwise → send. A held reply may be released at once by the module's rollout plan percent.
- Topic answers are only renewal / maintenance / other; leasing is decided by the recipient being a prospect.
### Callers of `shouldHoldClaraReply` (8, not 7)
`process-envelope.ts` (reply kind; routed kind), `process-inbound-email.ts` (routed), `lambda/inbound-processor/handler.ts` (routed, Graph), `process-pms-email.ts` (routed), `supervised-reply-hold.ts` (reply kind, email hook), `twilio/client.ts` (door), `company-live-gate.ts` (door).
### Other outputs
- F12 Ask for help (hand-off) | Opens an EscalationMatter and asks the owner (property then org chain) by email or a staff text | `src/lib/domain/escalation/open-matter.ts` : `openMatterAndAsk`, `openMatterAndAskOrHeld`, `openMatterAndAskCore`; owner `resolveEscalationOwnerViaChain`; tools `escalate_to_human` (`src/lib/tools/handlers/handle-escalate-to-human.ts`), `forward_to_property_manager` (`handle-forward-to-property-manager.ts`) | SH
- F13 Page the manager (urgent maintenance, gas/CO) | `src/lib/domain/maintenance/page-property-manager.ts` : `pagePropertyManager` | MA (core-consumed)
- F14 Answer comes back | email reply → `reply-release.ts` : `releaseMatterOnStaffReply` → `runTeamAnswerTurn` (relays to the person, teaches the knowledge base); team-notify answer → `team-notify/clara-turn.ts` | SH
- F15 Stuck ask follow-up | one nag via Temporal `escalationMatterWorkflow`; the cron `/api/cron/escalation-matter-nag` → `nag-pass.ts` now posts stuck matters to Slack alerts (`stuck-matter-surface.ts` : `surfaceStuckMatter`); the nag email is retired | SH
- F16 Alert engineering | `logCritical` → Sentry fatal (`src/lib/platform/logging.ts`); `notify({channel:'alerts'})` (`src/lib/platform/observability/notify.ts`); team-notify `alertEngineering` → Sentry | SH
- F17 PMS write via outbox | guest-card intents (create card, append note, cancel tour, mark inactive, assign, send application) → outbox → drain (on demand + cron every 15 min) → `PMSGuestCardWriter` | `src/lib/domain/pms/guest-card-text-note.ts`, `guest-card-outbox.ts`, `src/lib/data/dynamo/pms-write-intent-outbox.ts`, `src/lib/integrations/appfolio/guest-card-writer.ts` | LE (own writers in MA and RE)
- F18 Reminders | promise ledger after the turn (events bridge → trajectory grade after ~6h quiet → `promiseWorkflow`); PM action reminders (dark: off by default per company) | `lambda/conversation-events-bridge/registry.ts`, `temporal/activities/promise.ts`, `temporal/pm-action-reminder-triggers.ts` : `startPmActionReminderCadence` | SH
- F19 Dark (default off) outputs | team-notify engine and team-notify text (`src/lib/team-notify/setting.ts`), emergency alert call chain (`emergency.alert_chain`), PM action reminders (`pm_action_reminders.enabled`) | SH
## G. State written
- G1 Conversation and messages | `src/lib/data/store.ts` : `saveNewMessages`, `saveConversation`; `src/lib/domain/messaging/conversation-writer.ts` : `markReplyRowHeld`, `markReplyRowNotSent`, `backstampProviderIds`
- G2 Held drafts | `EscalationSmsHold`, `EscalationHold` (`HeldMessage`) | `src/lib/data/dynamo/escalation-hold.ts` : `saveEscalationHold`
- G3 Asks | `EscalationMatter` | `src/lib/data/dynamo/escalation-matter.ts`
- G4 Turn outcome | `recordTextOutcome` / `recordEmailOutcome` → `stampConversationTurnOutcome` on the conversation; inbound Slack card edited by `finishInboundCardOutcome`
- G5 Traces and snapshots | `saveAgentTrace`, `persistTurnSnapshot` (`dynamo/turn-snapshot.ts`)
- G6 Refusals and touches | `recordModuleRefusal` (`module-refusal-log/record.ts`), email log rows BLOCKED, `recordTouch`
- G7 Promises | `src/lib/data/dynamo/promise-ledger.ts`
- G8 PMS write intents | `src/lib/data/dynamo/pms-write-intent-outbox.ts`
- G9 Activity log | UNVERIFIED: no generic per-turn activity log (only `recordActivity` in `renewal-saga/writer.ts`)
- Everything lives in one DynamoDB table `propflow-data` (`src/lib/data/dynamo-client.ts`)
## Module concepts (shared vs own copy)
See figure `machine-modules`. Own copies per concept (count across LE, MA, RE, TC): automated-send review doors 3, topic classification 4, hand-off 2, person lookup 3, PMS writer 3, reminders/cadence 4, quiet hours 2, examples 4, tool sets 4, intake 2, scheduling 3, staff notice sender 3, module-off refusal 3, entitlement 3.
Invented: Leasing = tour pipeline, guest-card engine with PMS write-back, application-link chain, outreach cadence. Maintenance = work order intake and triage, vendor dispatch and chase, life-safety and on-call chain, handyman texts mode. Renewals = offer saga and packet, auto-start scanner, decisions engine, deferred tool calls under Supervised. Tenant comms = read-only resident answers, lease-terms answer engine, key pickup, unanswered-questions digest.
Shared pieces housed in a module folder: the hold's topic classifier and `holdDraftReplyText` (renewals files), the knowledge block (leasing file), `pagePropertyManager` (maintenance), scheduling intent engine (leasing). Two staff-notice "one doors" coexist: `escalation/team-notice-send.ts` and `notifications/human-notification-lane.ts`.
## Deltas vs the 2026-10-24 inventory
- `shouldHoldClaraReply` has 8 call sites (brief said 7).
- Daily nag email is retired; stuck asks now post to Slack alerts.
- Shadow modes are deleted, not drawn.
- Line numbers moved; this file cites functions only.
## UNVERIFIED list
1. Dashboard "reply as Clara" route (none found).
2. Human-replied gate at SES / Graph email send doors.
3. PMS text door's internal gate order (shares the compliance gate; order not traced).
4. Escalation nag cron schedule.
5. Generic per-turn activity log.
6. Memory store as a turn input.
7. Tenant PMS write from the Clara loop (`set_tenant_email`, may be staff tooling).
8. Knowledge on tenant turns traced end to end (marked shared from prompt text).
9. Prompt example counts are grep counts, not curated counts.
10. "Email Sent" PMS entry detail.
The four pages carried 24 open questions; the brain and platform pages a few more. Internal engineering is decided here; three owner calls survive (decisions for Fede). Rows that only restate standing rules (adapters per PMS, no confirm screens, phrases follow the desk) were dropped.
| Question | Decision |
|---|---|
| Where the staff side lives | One staff desk: every channel feeds one ask record routed by the ownership map; the six loops migrate one per PR behind an interface; no big-bang move; a hosted seat in the customer's tools comes after |
| A central home for the blocks (Fede: “maybe rip all this out into a central location”) | Option A, chosen: the interface is in place now and each module migrates one PR at a time as it reaches the same recipe shape; no code move until two modules share that shape unchanged. Option B, move everything now, rejected as a big-bang reorganisation with nothing to replay against |
| Engine or prompt decides the hand-off | Hybrid: engine owns the record, owner and routing; model owns the words; a checker blocks unbacked claims |
| Where the hand-off state lives | On the record that owns it (ask, work order, offer, ledger step). The ask record is the existing escalation matter (owner, waiting-on-staff, staff answer, last contact), tied to a person instead of an address, with end states answered / withdrawn / replaced / junk so pasted PMS notices stop sitting as open questions. The 17 ways Clara reaches staff today fold into it, stay as a channel, or are deleted; Figure 13 lists all 17 and names what is deleted (Constitution II.2) |
| "Office has opened since the ask" is a state | Yes |
| What counts as handled | Any activity by the owner on the thread moves the ask to picked up; it is answered only when its topic is settled (so "no worries, be safe" does not close a pet-limit question); never a clock |
| Situation deadlines vs banned clock rules | A deadline that belongs to the situation (tour start, emergency) is a state input; a timer on the hand-off itself is not allowed |
| Takeover | Moves the whole thread; a staff answer settles only its topic |
| Held drafts and supervised mode | Shadow-and-diff where a person already answers; the customer's own button (Send in AppFolio) where one exists, the only hold left; on Clara's own line and mailbox she answers covered topics live, sends an ask for the rest, and is corrected afterwards; nothing waits on us (Fede, Oct 21: "hold mode is the problem") |
| Readiness | Three numbers on the customer's page, each with a floor: coverage, hand-off correctness on replay, clean shadow diff (Constitution VII.4); PropFlow sets the floors, the customer may raise them; the values are question 1 |
| How we learn who owns what | Infer the doer; ask approver, CC and backup per topic; store as assignment rows resolved nearest-wins; the two new areas and the approver slot are proposed to Gera (bridge post after Fede's read), not built against his model |
| Ask for a cell at invite and accept | Yes, optional, nothing sent |
| Clara names the staffer to the renter | Role, not name, until the person has confirmed their row |
| How a brain item becomes public | A relayed answer is recorded on its ask; a standing answer becomes a public policy row, sorted by Clara with no question (Fede, Oct 24), and never for a floor exception like the three cats; everything else from meetings, mail, notes and connectors is internal until a person publishes it |
| What "covered" means | Answerable live from the brain on the office's real past questions, never "facts found" |
| Merge gate for the desk | A replay of 30 days of each customer's real traffic through the new rule (Constitution III.4) before any turn-on; the ~979-scenario harness is build hygiene, never the proof (III.6, no fake scenarios) |
| Intro trigger and per-person preference | Onboarding accepted and each new person on the map, once; preference from the reply and the staff page, stored as three data fields (channel, hours, detail) |
| Clara proposes PMS setting changes | Advice in plain words with an offer to do it; dark, company setting; never a silent break when declined |
| Autonomy levels | Two, by how much the customer's team approves before a commitment goes out (approve first; by policy with exceptions to people), taken from the published levels-of-automation scale, acting always the baseline; per module, the lowest of what readiness allows, what the tier allows and what Fede turned on; the tier is a ceiling (Fede, Oct 24 night; replaces talk, act, commit) |
| Pricing on this page | Fede's figures stay as the tie between levels and the price list, as he asked; both reviewers object (price on scope only, autonomy earned by readiness, never sold; billing tied to a safety control makes a demotion a refund conversation). Recorded in reviewer disagreements for his read; the pricing design itself is its own page |
| # | Way Clara reaches staff today | Call | What changes |
|---|---|---|---|
| 1 | Recognising staff | keep | one resolver for every door |
| 2 | Staff ask a question | keep | one tool list by role; per-door lists deleted |
| 3 | Staff tell Clara to do something | keep | habit E; one confirm rule |
| 4 | Staff teach a standing answer | fold into the ask | answer lands on the ask; standing answers become policy, sorted by Clara |
| 5 | Clara asks staff (four kinds of ask) | fold into the ask | one ask record, one sender |
| 6 | Hand-off and hand-back | fold into the ask | picked up / answered states |
| 7 | Tour check-ins and no-shows | fold into the ask | carries its tour and person ids |
| 8 | Lockbox codes | fold into the ask | a remembered code carries an expiry |
| 9 | Key pickup | fold into the ask | an ask or a notice through the same record |
| 10 | Transfers, paging, callbacks | fold into the ask | a missed transfer becomes a checked callback |
| 11 | After-hours emergencies | keep | emergency mode (floors); acknowledgement tracked |
| 12 | Work-order vendor approvals | fold into the ask | approval by reply; review page deleted |
| 13 | Staff notices (about 20 templates) | keep | one notice engine; one-off senders deleted |
| 14 | Supervised-week holds | delete | no hold on Clara's own channels (supervised) |
| 15 | Teammate already acted in the PMS | keep | one staff-touch check before every send |
| 16 | Renewals list to the team | fold into the ask | the monthly list uses the same ask |
| 17 | Meetings and video | keep | a door; notes become asks to confirm |
Lifecycle side: four Opus 5.5 digests of 29 past pages, the Oct 24 hand-off audit (810 rows, counts as corrected by the Escalations session's claim check), the Oct 16 staff-loop queries, the Oct 21 Western Slope working-session transcript, Fede's Oct 22 to 25 notes as relayed through Trinity, a Gera-style study of his audit and architecture pages, and two adversarial reviews. Full text of the four pages absorbed earlier is parked under ~/agents/006/parked/. Internals side: the Benchmark lane's read-only inventories and benchmarks as cited in each table, and its fragment of decision surfaces and ledgers, carried over verbatim. This rewrite merged Clara, the new hire and Clara re-architecture: the one page to review on Oct 25.
#decisions-benchmark, because #decisions is the whole decisions section. Heading levels of the carried decision blocks moved one level down to sit inside section 8; their wording and numbering are unchanged.Fede, Oct 24: Clara will carry every office module in layers (Clara, the new hire). One giant prompt on a mid-size model cannot carry that. This lane puts down the foundation: specialized small prompts per job, a frontier model under them, and a repeatable safe way to upgrade the model as new ones ship. Everything else on this page is a tactic under those three.
| Done means | Today (Oct 24, 19:30Z) | Gap |
|---|---|---|
| 1. Today's prompts decomposed into job-specific pieces, same voice and rules, in production | First reply split into four pieces on a local branch. Rerun Oct 24 evening (72 cards, 3 runs, judge majority): lookups skipped 10 to 6 (today's prompt on Opus: 3), no-reply 2 to 0, Spanish 4 of 4, no dashes, all four home facts on 31 of 68 (production 8), tone match 46 of 68, full match 27 of 68. Rerun 2 (Oct 24 ~22:30Z): dropped answers 12 to 2 (production 8), skips 3 (production 18), Spanish 4 of 4, no dashes; on the four graded examples the arm now reads as good or better (#27 answers the voucher question the way Fede graded it). Rerun 3 (Oct 24 ~23:30Z, both arms run in the same hour so live inventory matches; 69 cards): dropped answers 10 to 4, lookups skipped 20 to 5, dashes 61 to 0, tour window kept 18 to 57, all four home facts 7 to 32, no reply 1 to 0. Graded four: #26, #27, #30 at parity or better; #25 answers more but recites the pet-approval procedure (fix line queued). Spanish regressions were stale inventory, gone in the same-hour run; Spanish naturalness 17 vs 17. Match judge re-taught (full match 15 to 25 of 69), planted controls 32 of 32 caught. Repair side: fixes land, duplicates double because nothing matches a text to an open work order. Rerun 4 (Oct 25 ~03:00Z, 66 pairs, ten fix lines, same hour): skips 17 to 3, dropped answers 11 to 2, dashes 56 to 0, four home facts 9 to 30, tour window kept 25 to 52, Spanish "Hola" 9 to 12 of 12, turn time 38 s to 26 s; graded #25 and #30 better, #26 same, #27 same (better in the two prior runs). Parity bar NOT met: in 4 of the last 5 runs one draft was a note about the prospect ("I've sent Jed his first reply") that no check on main blocks; would-edit-by-a-rewrite-layer 18 vs 8 (honest "not on file" lines and extra facts tripping the fact check); the exact ten-line prompt has run once. Branch pushed, no PR. | Decided under the grant: (1) a blocking check for a first email that reads as a note about the prospect, own small PR first, plus the writer told it writes to the prospect; (2) fact check tuned so honest "not on file" lines and on-file facts pass; (3) then the prompt is frozen and run 3 times in the same hour; the production PR (decomposed first email on Opus 5.5, no switch) opens only if all four graded cards hold in all 3 runs. |
| 2. Clara on a frontier model (Opus 5.5) in production | Light-mode runs only. The six "worse" cases read (Oct 24 19:5xZ): 3 of 4 prospect ones are a bench artifact (the bench stores Sonnet's draft before production's rewrite step edits it); the 2 repair ones are real and the same habit: Opus hands a clear repair to the team instead of filing it (team items 50% to 64%), and splits one report into several orders more often (2 of 54 to 6 of 57). The 15 "improvements" are dash cleanup only. Spanish: Opus answered in Spanish 54 of 54, Sonnet 49 of 54. Invented facts about half as many on Opus. Production bug found on either model: the inserted "I'll check with the team" line comes out in English inside a Spanish email. Nothing blocks the model change; the two habits are prompt work. | Full-mode proof (running now, repairs first, emails after PR 22238 deploys); fix the two habits in the decomposed repair prompt; fix the English-line insert; store what production actually sent as the reference arm. |
| 3. A repeatable upgrade: replay real episodes through the real path, production as reference, judge plus Fede's grades, history over time | Full mode built Oct 24 (PropFlow PR 22201 armed, benchmark PR 15 merged, PR 16 armed): seven read tools run for real against production inside a per-request read-only scope that refuses every write; every other tool is blocked by name and returned with its input. Proven on 5 recorded Willows tenant texts from this machine: the work-order lookup returned the real orders, 2 side writes refused each, a forward-to-staff call blocked, a planted close-order call blocked. Gaps: the door builds tenant text turns only (prospect emails refused; builder on it), repair-episode ids are AppFolio text ids not conversations (resolver in the same build), reads are as of today not of the turn, one recording the model API rejects (issue 22213). Recorder live since 15:13Z: 1 recorded turn at Western Slope, 0 at Situs. Proven in production Oct 24, 23:14Z (PR 22201 live in 1503b6562): on the 11 recorded Willows texts the door ran the real work-order, balance and lease lookups against live records, refused 2 side writes per lookup, blocked a note-to-team send and returned its full input; a before/after diff of 38,511 Willows keys shows zero new conversation, message or work-order rows. First full-mode numbers (11 turns, direction only): Sonnet 5 light 0/11 → full 2/11; Opus 5.5 light 6/11 → full 8/11. Opus answered "when is the tech coming?" from the real order. Those Opus numbers were production's prompt on Opus, not the decomposed prompt (the runner could not apply it until Oct 25). Opus full-mode misses: promises "the team will take it from here" before any repair is filed and forgets to ask which unit (3 turns). Fixed Oct 25 in the decomposed repair prompt (two rule lines, local branch, no PR): Willows 11 turns Opus full 8 → 9 pass, light 6 → 8; replies saying the team has it before a work order 6 → 0 (full) and 5 → 0 (light); all three bad turns now pass. G3 261 episodes: team-has-it 2 → 0, invented 16% → 15%, duplicates 10 → 8, filed 24 → 22 of 42 (dispatcher step, noise), Spanish 34/34, judge 4.01 → 4.01. Decision made under the grant (Fede can override): on a safety emergency Clara reports to the team at once before the unit is known and asks the unit in the same reply, as production does today. Added and rerun: Willows Opus full 8 → 10, light 6 → 8, team-has-it 0; the 2 AM leak turn now alerts the team, gives the shutoff step and asks the unit in one reply. G3: filed 24 → 25 of 42, invented 43 → 41, promises outside her tools 22 → 18, duplicates 10 → 9, team items 58 → 56 (churn is routine both ways, not from the line), judge 4.01 → 4.08. Match-before-filing rule tried Oct 25 (versions 3c to 3e, local): duplicates 9 → 7 of 93, but 4 of the 7 are orders that exist only as a number in the conversation, which production never sees, so 3 real duplicates remain (a stove request against a roach order, a washer against a dryer order, a kitchen fan); filing fell 25 → 19 of 42 and one wrong attach stays (a bedroom AC leak added to the outside-spigot order). Willows 11 turns: full 9/11, light 10/11, team-has-it 2/0. Not accepted. A code check (file new when the matched fixture is not named in the open order) was tried as 3f and missed too: real duplicates 7, filed 22 of 42, the AC-leak case still attaches to the spigot order because intake copies the order's fixture name. Stopped there. The accepted version stays the two rule lines plus the emergency clause (3b: duplicates 9 of 93, 3 of them real judgment calls; filed 25 of 42; judge 4.08). Matching a new report to an open order needs a design, not another prompt line; the 3 real duplicate cases and the AC-leak case go in Fede's review set so his verdicts set the bar. Door fix Oct 25 (PR 22351, live 2:35 AM MT): a tenant text the inbox mirror filed onto a call or email thread now counts as a text by the message itself; 7 of the 15 Western Slope refs from issue 22320 now load, the other 8 are refused only because the person the thread is filed under is not an active tenant at that home (possibly the mirror filing under the wrong person; noted on the issue, not fixed here); 0 Slack posts from the run. Full-mode repair comparison done Oct 25 on the 20 runnable episodes (see the bench section above): decomposed on Opus 4.00 vs today 2.85 on the reply; filing and duplicates still unmeasurable in full mode because the door skips the filing step. Gaps: who-is-texting tool not on the read-only list (issue 22285); repair episodes refused until PR 22238 deploys (0 of 261 runnable today, not 19); 8 of 11 turns read records as they are today. | Merge PR 22238, rerun the repair tally, then repair and prospect-email runs on both models in full mode. |
How every Clara change is benchmarked and released (proposed): Clara Bench page, section 6b: what we already have for each stage, and the gaps.
The three fixes shipped Oct 24 (first-reply opener, four prospect-email fixes, work-order number) are proven live and are not what done means.
Built Oct 24 19:3xZ from the session transcript after Fede: "ive already asked for all of this multiple times". Relayed asks count as asks.
| Asked | Ask | Status | Evidence or gap |
|---|---|---|---|
| Oct 24 | Benchmark loop against the production door must not touch ops or Slack | Fixed, proven in production | Oct 24, 8:56 PM MT: an ids-only benchmark loop against the production bench door posted 96 identical 'tour calendar unavailable' ops cards to Western Slope's Slack in 47 seconds. The door blocks record writes and sends but not ops cards or Slack notices, and the once-a-day dedupe fails open when its marker write is refused. No text, email or call went to anyone, no record, AppFolio or calendar write, no setting changed. All production door runs are stopped until two fix PRs are live (door suppresses every ops and Slack side effect; dedupe treats a refused write as 'do not post'). Proof when done: the same loop rerun with zero new cards. Update 10:20 PM MT: the host's calendar was never really down; the bench's read-only mode caused the failed read (the Google login was refreshed but the new token could not be saved). His login is intact, refreshed and saved by production at 10:00 PM MT and marked connected, so Western Slope tour booking was not affected. Fix PRs, all three open and armed to merge on their own at 10:25 PM MT: #22324 (door posts nothing), #22327 (dedupe treats a refused marker write as 'do not post'), #22329 (a calendar token refreshed under read-only mode is used in memory, never saved); incident issue #22323. All three merged and live by 12:15 AM MT. Proof 12:30 AM MT: the same 72-card loop rerun against production: 0 new cards in Western Slope's Slack (the incident run made 96), 117 writes refused, and the host's calendar read back real tour times under read-only mode. Door runs are unfrozen. Issue #22323 closed. |
| Oct 22 | First-reply deep analysis: prompt inventory, conflicting rules, cold cache, tool list, effort settings | Done | Architecture audit page |
| Oct 22 | Never say how many homes are open, no rosters, no future openings | Done | Count and roster check on email, facts block, voice (3 PRs merged Oct 22-23) |
| Oct 22 | Tour confirmation by text whenever we hold a number, email too | Done | 4 PRs merged Oct 22-23 |
| Oct 22 | One shared line resolver, hard-coded map deleted | Done | 5 PRs merged Oct 22-23 |
| Oct 22 | Prove a prospect texting back through AppFolio texting gets a correct Clara reply at a real customer | Proven with defects (Willows, Oct 24 19:30Z) | Read Oct 24 20:0xZ: 15 Clara texts out on the AppFolio line at Western Slope since Oct 22, zero prospect replies back, so the reply half has never run at a customer; pick-up works (read in 11 to 59 s, 5 prospect texts correctly left alone because the office spoke last). Willows live run 19:30 to 19:38Z: four prospect texts to the AppFolio number, four Clara replies from that same number in 49 to 103 s, nothing on her own line, no setting changed. Two real defects found: (1) the prospect said "Saturday at 3:00", the tour step booked next Saturday Oct 31 while Clara texted "today, Saturday, October 24th", twice; (2) when Kat texts Clara her post-tour notes the copy saved on the prospect thread looks like a teammate message, so the prospect's next reply counts as "staff spoke last" and Clara stays silent with no alert (one Western Slope thread, Oct 24 17:00Z). Also em dashes in Clara's text replies. Fix builders started for all three. Side finding: 69 of 109 texts on Western Slope's AppFolio line since Oct 15 were never filed onto any conversation (39 matched a future tenancy, 29 a person with two active tenancies); none were prospect replies. | Update 21:22Z: the staff-note defect is fixed and live (PR 22218, deployed 21:18Z). Same two texts at the Willows before and after: before, the wake check read the teammate's note as "the office spoke last" and Clara stayed silent; after, Clara answered the prospect in 87 seconds from the AppFolio line. Replay of 444 prospect messages on 86 Western Slope threads with such notes: 10 were misread this way, none of them on the AppFolio line, and the two threads where staff really answered still keep Clara silent.
| Oct 22 | 'Draft held, not sent' row must never show for a sent turn | Fixed, proven in production | PR #22208 merged Oct 24 and reached production 23:08Z. Before: the Ember Estates thread showed one 'Draft held, not sent' card above the email Clara sent 18 s later. After: no card; the thread reads lead, email, text, two follow-ups. QA on the real page: neighbouring thread unchanged, details pane opens, conversations list (79 rows) pages normally. Screenshots on the PR. A draft that truly never got answered keeps its card: unit-covered only, no live example found in ~450 recent threads. |
| Oct 24 | Production release went red (marketing function 416 MB, deploys blocked ~3 h) | Resolved | #22270 ([urgent]) fixed the bundling; production serves 1503b6562 since 23:08Z, which also carries the full-mode door, the no-day-asked fix and the NEW LEAD card fix. My duplicate #22268 closed. Root cause tracked in #22262. |
| Oct 22 | Latency per segment and a target floor | Not on the page | Analysed in chat Oct 22; not written up |
| Oct 22 | Guest-card notes: outcomes not turns, hand-rewritten before/after, then build | Done | 3 note PRs merged Oct 23 |
| Oct 23 | Delete the four dead rewrite layers | Done | Merged Oct 23 |
| Oct 23 | No rewrite layers at all; checks only block | 4 of 4 PRs merged; first email witnessed live at Western Slope Oct 25; follow-up text writer not yet witnessed (the 8 AM Willows touches on harness leads failed before writing because their numbers are fake 555 numbers, and one lead has no phone at all; next real chance is the Oct 26 8 AM touch to the bench lead) | Cut by path, each a fix with no switch. A: first email, checks only block, one writer retry then hold (PR 22249, merged Oct 24). Lock-in false match that cut a Willows email mid-sentence (PR 22292, merged Oct 25 [urgent]). C: follow-up texts, same rule, fair-housing flag-and-post on follow-ups (PR 22306, merged Oct 25 03:5xZ; replay 117 pairs: Spanish threads answered in English 11 → 1, held turns still sending something 3 → 0). B: main-loop text and email (PR 22296) in review; after two 🔴 on the retry re-check it was redesigned as one send gate every reply passes right before send; pushing. D: post-call notes, stale-draft editor and dead paths, next. Follow-ups: #22301 (availability-disclosure block-only reason), #22310 (outage page copy), template greeting stays until the writer reliably greets. |
| Oct 23 | Re-inspect every gate: working or not | In progress | Inspector running (started 19:05Z) |
| Oct 23 | Sonnet 5.5 columns, cost section, Opus price projection, caching opportunities | Not on the page | Runs finished and answered in chat; architecture page has no cost or cache section |
| Oct 23 | Tenant leak: former tenant folded into tenant guardrails, unknown role never replies, find other leaks | Done | Fix merged Oct 23; leaks page |
| Oct 23 | Staff-typed guest card never gets Clara | Held by you | You said the implementation was never reviewed; nothing shipped |
| Oct 23 | Pull the raw Gera transcript, rewrite the plan page | Done | Plan page rewritten Oct 23 |
| Oct 23 | Prototypes and before/after benchmarks on thousands of real records, today's pipeline vs decomposed | Partial | 72 prospect intents, 261 repair episodes, 837-card writer run, 114 tenant pairs; full mode (real tools, writes blocked) merged for repair texts (PR 22201, live 23:08Z) and for prospect first emails plus repair-text refs (PR 22238, merged Oct 25 01:1xZ, deploy pending); first production full-mode run done on 11 Willows turns (section 0 row 3); repair and prospect runs on both models start when 22238 deploys. Limit found: only 19 of 261 repair episodes map to a text the mirror filed (215 are history-export rows the mirror never saw), so the repair set in full mode is small until the export rows are loaded |
| Oct 23 | Tenant comms in supervised mode by next week: architecture, volume, cost, when to stay silent, digest | Partial | Section 8 has architecture and volume; no cost line; nothing built toward turn-on |
| Oct 24 | Review set: full context per card (open work orders, prior thread), real names | Done | Context and names added Oct 24 |
| Oct 24 | No team items in phase 1; silent or helpful; unanswered goes to a daily digest | Ruled | Rule applied in the when-to-reply prototype (D3); not in production |
| Oct 24 | One work order per tenant report, not one per issue | Ruled, not shipped | In the decomposed repair arm only; production still files one per issue |
| Oct 24 | Acknowledge every item in a text (renewal interest, call request) beside the repair | Ruled, not shipped | Same |
| Oct 24 | Shared-area problem files on the property, not the unit | Ruled, not shipped | Same |
| Oct 24 | Misspelled Spanish repair text must not be treated as silent | Open defect | Seen in the review set; not fixed |
| Oct 24 | Check photos reach Clara and whether she can reason over them | Partial | Photos in AppFolio text threads reach Clara and the work order (merged Oct 24); vision reasoning not checked |
| Oct 24 | Tenant hears the PMS work order number | Done | Merged and proven live at the Willows 18:27Z |
| Oct 24 | Commercial work orders: research and a commercial sub-prompt | Research only | Noted on the page; no research delivered |
| Oct 24 | Vendor scheduling design doc, DM when done | Done | Page published, DM sent |
| Oct 24 | Benchmark app, ingest style, separate repo, snapshots recorded in production | Built, UI rejected | Recorder and door live; UI redesign proposed in section 11, waiting on you |
| Oct 24 | First-email opener, four email fixes | Done | Merged Oct 24, proven on replay |
| Oct 24 | Decompose the prompt (same voice), run on Opus, as good or better on every graded example | Not done | Skips 11 to 6 of 72; worse than production on 2 of 4 graded examples (dropped answers); builder on it, local branch |
| Oct 24 | Tour date mismatch: Clara said "today", the tour landed a week out | PR 22233 merged, move path proven live | Cause: a bare "Saturday" said on a Saturday was read as next week because Clara's same-day offer never reached the tour pipeline. Fix: the day is read from the offered slots, and Clara's confirm and move wording comes from the tour record. Replay of 2,156 real replies: exactly one changes day (the Willows one, Oct 31 to Oct 24). Willows live move at 22:11Z: "move my tour to Monday at 11" moved tour 2ff2c982 from Sat Oct 31 to Mon Oct 26 11:00 AM, and the tour record, the team email and Clara's text all say Monday, October 26 at 11:00 AM. The same-day case (offer today, prospect answers with the bare weekday) is proven on the next open day; the step is on the PR. Follow-up #22230: block any reply whose tour day disagrees with the record. |
| Oct 23 | No em dashes in any Clara writing (texts, follow-ups, staff notices, repair texts) | Merged, proven live at Willows | Before: 188 of 440 sent messages in 14 days had a dash (text and email replies 69%, staff notices 28%, follow-ups 10%). One shared rule and check across every written path; a reply with a dash goes back once, then the team is asked. Replay of 30 real messages: 20 had a dash before, 0 would go out with one now (2 held on an empty rewrite). Fixed-wording staff and vendor texts are follow-up #22237. Live Oct 25 00:30Z. Willows proof 00:32Z: a repair text and a prospect text each got one clean reply (0 dashes); on the prospect reply the check caught an em dash in the first draft and the redraft went out clean (Twilio SMa9a0d9eb, SM19251428; messages msg_667b3009, msg_3bd05319). Western Slope before: 7 of 19 Clara messages in the prior 24 h had a dash, including Lori's tour confirmation; after: no real message sent yet (Saturday night), tally reruns Sunday. Leftover: the staff tour booked/rescheduled email template still carries em dashes in subject and first line, added to #22237. |
| Oct 24 (Lori, Pomona Park) | Why "weekend tours need to be booked by Friday"; why a text too; why no Slack new-lead card | Three fixes merged and live; (1) proven at Willows | (1) No date asked, prompt checks "today" (a Saturday), the weekend cutoff rule returns a blocked-day sentence, and a prompt rule makes her explain it: fix = never explain a blocked day nobody asked for, PR 22241 armed. Replay of 54 real first replies since Oct 8: 9 explained a blocked day before, 1 after (that prospect asked "today or tomorrow?", so correct). Lori's card replayed: "The soonest I have open is Monday, October 26th..." with no weekend line, 0 of 6 runs vs 4 of 6 before. Willows email run after deploy. (2) No text reached her: the "I emailed you, text me here" note was held for a missing guest card and gave up after 10 tries in 10 minutes; the card arrived at 14 minutes: fix = held texts wait up to 6 hours. PR 22240 merged and live in the sync service (23:1xZ): a text held for a missing card now retries at 1, 2, 4, then every 5 minutes up to the 6-hour cap, with one alert only if it expires. Replay of 7 days: 3 held texts, Lori's would have gone out 252 seconds after her card, 0 double sends. Already-dead texts stay dead (nothing resent). Live proof: the next real Western Slope lead whose card arrives late (watch running); both bench numbers already hold Willows cards so they cannot stand in. (3) Since the Oct 22 one-card-per-lead change, leads Clara creates herself (Zillow email) get no NEW LEAD card; 3 so far: fix = post the card at email intake, PR 22239 armed (7-day replay: the 3 missed leads get 1 card each, the 10 PMS-created leads unchanged, no lead gets two; overlaps Gera's #21824, Smith told). Proof is the next real Western Slope email lead after deploy; Willows rolls cards into a daily summary so it cannot show the per-lead edit. All three merged and live by 23:08Z, no switch, one PR each. (1) proven live at Willows Oct 24 23:34Z: a renter wrote "I would like to schedule a tour" with no day; Clara's draft read "the nearest opening I have is Monday, October 26th, anytime between 9:00 and 9:30 AM, or between 11:30 AM and 4:30 PM" with no weekend line (conversation 894a2288, message msg_ce40c7bc, evidence on PR 22241). The email that actually went out was cut off mid-sentence by a leftover rewrite step (the "tour already booked" check matched "lock in" in the lease special and the rewrite came back truncated): not caused by this fix, issue 22289, next PR in the rewrite-removal chain. (2) and (3) wait on the next real Western Slope late-card text and email lead (watches running). DM sent to Fede 14:3x MT. |
| Oct 24 | Are we ready to upgrade to Opus 5.5 | Not answered | Light mode plus one full-mode run on 11 Willows turns (Opus 8/11 vs Sonnet 2/11, but Opus over-promises on repairs); the repair and prospect sets in full mode wait on PR 22238 |
| Oct 25 | Slack rating buttons on the shadow cards, one label record feeding the bench (Fede's go via the Trinity DM) | building: app PR 1 and harness PR 2 in flight, card buttons on the shadow runner after they merge, bench read PR 3 after PR 1 | Human labels on the Clara Bench page |
In one line: the Benchmark is now its own app, PropFlow gained two small pieces it needs, and both benchmarks ran with production as arm 1.
| Measure | Production today | Same prompt, Opus | Opus + your fixes |
|---|---|---|---|
| Filed when the office filed | 45% | 48% | 57% |
| "Check with the team" | 34% | 39% | 8% |
| Invented facts | 30% | 21% | 16% |
| Duplicate of an open work order | 5% | 5% | 11% |
| Judge score (of 5) | 2.98 | 3.23 | 4.01 |
Only 20 repair texts can run with Clara's real lookups against production: 215 are Situs history texts older than the message mirror (which starts Sep 29) and were never stored, 15 are held by the mirror, 10 come from a sender who is not an active tenant there, 1 has two tenancies. Until the evening, full mode could not measure whether Clara filed a work order when the office did, because the door skipped the filing step; that is fixed and measured below. On the reply itself, same 20 texts, both writers in one pass, judge controls 4 of 4:
| Measure (20 texts) | Today's Clara (Sonnet 5) | Decomposed on Opus 5.5 |
|---|---|---|
| Judge overall, out of 5 | 2.85 | 4.00 |
| Invented facts | 8 | 4 |
| Passed it to the team | 10 | 2 |
| Promised something outside her tools | 7 | 0 |
| Stayed out when the office was running the thread (of 10) | 0 | 8 |
| Added a note when an open order covered it (of 6) | 4 | 2 |
| Clear repair report left silent (of 5) | 0 | 1 |
| Note put on the wrong old order | 1 | 0 |
Filing, measured Oct 25 evening. The door now runs production's own work-order filing step read-only (merged as #22386, live, proof: 0 writes, 0 notices, 0 Slack cards). For each of the 20 texts, this is what today's production filing logic would do, next to what the office actually did. The office column comes from the G3 records matched on each text's source id.
| Text | Production filing step would | Office actually did | Read |
|---|---|---|---|
| Situs 1158429018 | reply only | filed a shower faucet order | miss |
| Situs 1166455347 | reply only | filed a dryer order | miss |
| Western Slope 1235853322 | reply only | filed an other-maintenance order | miss |
| n51 | reply only | filed a bathroom flooring order | miss |
| Situs 1159587096 | new order: person approaching residents | filed a shower faucet order | different problem |
| n52 | two new orders (washer, dryer) | nothing | split report, against the one-order rule |
| Situs 1166988363 | new order: no water in unit | filed a water outage order | match |
| Situs 1161429708 | new order: laundry machines | 1 open order already | plausible |
| Situs 1167466911 | new order: duct cleaning | 2 open orders already | plausible |
| Situs 1156292270 (key not delivered) | add to the open security camera order | a replacement-keys order was open | wrong attach |
| 7 texts (Situs 1162814418, 1164004510, 1165635254; Western Slope 1234038727, 1234207333, 1236401104; n59) | add to an open order | open orders already existed in 6 of 7 | plausible |
| 3 texts (Situs 1161301929, 1164229225; Western Slope 1234562958) | reply only | open orders already or nothing | plausible |
Totals: add to an open order 8, new order 5, reply only 7. Against the office: 1 clear match, 4 misses where the office filed and production would only reply, 1 different problem, 1 split report, 1 wrong attach. Not run read-only: the actual create, the which-home rule, the emergency content check, the vendor-first check. Order ids are stand-ins, nothing was created.
Filing, decomposed Opus 5.5 vs production, measured Oct 25 evening. Same 20 texts, same door. The decomposed column is the accepted prompt (3b) with both halves on Opus 5.5: the four graded edits on the filing step's classifier prompt (one order per text, a worsening report updates the open order, a repair report is never reply-only, a leak is urgent), then the decomposed writer with the real lookups, seeing what that filing decided. In the G3 benchmark 3b ran the classifier on production's model (Sonnet 4.6) and only the writer on Opus; that exact version is the sensitivity line under the totals. Nothing was created or sent: 0 refused writes, 0 notices, 0 new messages in #western-slope; the writer's 5 notes and 1 team alert were blocked at the door. Read legend: match = filed what the office filed; miss = the office filed and this arm did not; different problem = filed something other than what the office filed; split report = one text, several orders; wrong attach = put on an open order for a different thing; plausible = no office order to compare, the choice is reasonable.
| Text | Production filing step would | Decomposed Opus would | Office actually did | Read: production | Read: decomposed |
|---|---|---|---|---|---|
| Situs 1158429018 ("I'll send it at 4") | reply only | reply only; writer stayed silent (office was handling it) | filed a shower faucet order | miss | miss |
| Situs 1166455347 (car and parking spot) | reply only | reply only; writer stayed silent (reply to the office's parking message) | filed a dryer order | miss | miss |
| Western Slope 1235853322 ("That has been fixed") | reply only | reply only; writer stayed silent | filed an other-maintenance order | miss | miss |
| n51, Spanish ("se lo comentaré a los que viven conmigo") | reply only | reply only; writer stayed silent | filed a bathroom flooring order | miss | miss |
| Situs 1159587096, Spanish (homeless people near the property, trash) | new order: person approaching the property | no order (the classifier's own check asked first); writer alerted the team and told her "ya alerté al equipo" | filed a shower faucet order | different problem | miss, plus a team alert on a non-emergency |
| n52, Spanish (washer and dryers in two laundry rooms) | two new orders (washer, dryer) | one new order naming both rooms and every machine; writer confirmed it in Spanish | nothing | split report, against the one-order rule | one order, as ruled |
| Situs 1166988363 (still no water) | new order: no water, emergency | new order: no water after the shutoff, emergency | filed a water outage order | match | match |
| Situs 1161429708, Spanish (laundry machines moved, dryer hose off) | new order: laundry machines moved | added to an old open washer order at her unit (ZEPH-15392); writer added the note there | 1 open order already | plausible | wrong attach likely (old washer order, different problem) |
| Situs 1156292270 (new door key not delivered) | add to the open security camera order | no order, asked which open order; writer asked if she is locked out | open orders already, incl. a replacement keys order | wrong attach (security camera order) | plausible (no note on the keys order) |
| Situs 1162814418 (electrical dispute) | add to a closed electrical order | reply only; writer stayed silent (office handling the dispute) | nothing | plausible | plausible, matches the office |
| Situs 1164004510 (washer needs replacing, unit 8656) | add to an open order and ask | no order, asked first; writer asked which order or a new one | 2 open orders already | plausible | plausible |
| Situs 1165635254 (valve replaced, 1144) | add to the washer installation order | no order, asked which order; writer added a note to the washer installation order anyway and asked | 2 open orders already | plausible | plausible (note before the answer) |
| Situs 1167466911 (duct cleaning at 1138) | new order: duct cleaning | new order: air duct cleaning | 2 open orders already | plausible | plausible (unit unclear, not asked) |
| Western Slope 1234038727 (living room light, needs an electrician) | add to the lighting order | add to the lighting order; writer added the note | open orders already | plausible | plausible |
| Western Slope 1234207333 (fridge to be replaced) | add to the freezer order | add to the freezer order; writer added the note | open order already | plausible | plausible |
| Western Slope 1236401104 ("The leak does seem to be fixed") | add to the leak order | add to the leak order; writer stayed silent (office asked) | open orders already | plausible | plausible |
| Situs n59 (two dryers, one screeching) | add to the dryer order | add to the dryer order; writer added the note | open order already | plausible | plausible |
| Western Slope 1234562958 (oven fine, rent late) | reply only | reply only | open order already | plausible | plausible |
| Situs 1161301929 ("happy Friday") | reply only | reply only; writer stayed silent | open orders already | plausible | plausible |
| Situs 1164229225 (unit turn status) | reply only | reply only; writer stayed silent | nothing | plausible | plausible |
| Totals (20 texts) | Production (Sonnet 4.6 classifier) | Decomposed (Opus 5.5) |
|---|---|---|
| Add to an open order | 8 | 5 |
| New order | 5 (6 orders) | 3 (3 orders) |
| Ask first (which order, or the split check) | 0 (1 as a side item) | 4 |
| Reply only | 7 | 8 |
| Match the office | 1 | 1 |
| Miss (office filed, arm did not) | 4 | 5 |
| Different problem | 1 | 0 |
| Split report | 1 | 0 |
| Wrong attach | 1 | 1 |
| Plausible | 12 | 13 |
Better with the decomposed arm: the two-laundry-room text (n52) is one order instead of two, which is Fede's one-order rule holding; the key complaint (1156292270) is no longer put on the security camera order (production's choice, which the earlier read had called plausible); the electrical dispute (1162814418) gets silence, like the office, instead of a note on a closed order. Worse: the laundry-machines text (1161429708, Spanish) goes onto an old washer order at her unit instead of a new order; the homeless report (1159587096, Spanish) files nothing and the writer sends a team alert and says "ya alerté al equipo", though it is not a safety emergency, so it breaks the no-team-items rule; the valve text (1165635254) gets a note before Clara knows which order. On 1159587096 the office's order is the shower faucet order (ZEPH-18587) that 1158429018 also maps to, so neither arm could have matched it from this text. Same on both: the no-water emergency (1166988363) files at once as an emergency, and the 4 shared misses are texts that are not themselves repair reports (a photo promise, a parking reply, "fixed", "thanks"), so the office's order likely came from another message. The four "ask first" rows come from Opus giving lower confidence than Sonnet (0.5 to 0.6 against 0.7 to 0.8 on the same texts) meeting production's fixed thresholds: a new problem with any open order needs 0.8 or it asks (1159587096, 1164004510), and a pick with no real order id asks (1156292270, 1165635254). The open lists make this worse: at Situs the door shows 8 to 43 open orders per tenant, most of them years-old "Work order (AppFolio)" rows still marked dispatched. So moving the classifier to Opus needs its thresholds re-set, or the open list cleaned, or both. No tenant in these 20 has two units, so the which-unit rule was not exercised. Sensitivity, 3b exactly as benchmarked (same edits, classifier on Sonnet 4.6): 5 add, 4 new, 1 ask, 10 reply; it also files n52 as one order, files the homeless report as a new order, and asks on the laundry-machines text instead of the wrong attach. The "I've put in a repair request for this." opener on 1166988363 and 1167466911 is the work-order confirmation production's finish step adds, not Opus (it doubles Opus's own line on 1166988363); the Spanish offer line on 1167466911 follows the company voice rule for limited English.
One caution on the four "miss" rows shared by both arms: those texts are not repair reports ("I'll send it at 4", a parking reply, "fixed", "thanks"); the office order on the same day came from somewhere else, so neither arm should have filed from the text. They count as misses in the table only because the office outcome is matched by day, not by text.
What this says for the lane goal: on filing, the decomposed prompt on Opus is not yet at parity with today's production. It fixes the split report, the key-on-camera attach and one needless note, but it asks first on 4 texts where production acted (Opus reports lower confidence and production's fixed thresholds turn that into a question), puts the moved laundry machines on an old washer order, and on the person-approaching-residents text alerts the team instead of filing. Both of the new misses are Spanish texts. Decision for the lane, not for Fede: the filing thresholds and the open-order list (8 to 43 stale open orders per Situs tenant) are tuned for Sonnet and must be re-set before the classifier moves to Opus; the sensitivity line (classifier stays on Sonnet, writer on Opus) is the safer first step and is what G3 measured.
One miss only full mode could show: with the balance lookup running for real, the decomposed writer quoted a tenant's two-week-old balance when she had not asked. Small sample; the light-mode numbers for the same 20 sit beside these in the report file.
| Check | Production (Sonnet 5) | Same prompt, Opus | Split prompt, Opus |
|---|---|---|---|
| Dash in the draft | 55 of 68 | 13 of 68 | 0 of 68 |
| Full home facts (beds, baths, size, rent) | 7 of 68 | 4 of 68 | 30 of 68 |
| Tour times offered | 54 of 61 | 60 of 61 | 61 of 61 |
| Skipped the first lookups | 19 of 72 | 4 of 72 | 11 of 72 |
| Piece | What it does | State |
|---|---|---|
| Recorder (PropFlow PR 22151) | Saves the exact context Clara was given on every turn | live, proven first real turn recorded Oct 24, 12:44Z: a Camellia tenant email, 336 KB of context (property facts, both model calls, the tool call, the 9 checks, the reply). 6 Camellia email turns and 8 Willows texts recorded today. |
| Service door (PropFlow PR 22152) | Lets the Benchmark build a turn's context or run one tool, sends blocked | merged own token live (PR 22173); replay from a recorded turn: live, proven 15:13Z (PR 22179): the door handed back the exact recorded context of a real Camellia turn, source "recorded", 248,523 characters of instructions, 26 messages, 22 tools |
| Benchmark repo: store and CLI | Keeps runs and grades; command line | on main |
| Benchmark web pages (Benchmark PR 8) | Runs, coverage and grading in a browser | merged, deployed 14:40Z; afternoon: regressed counts now computed from results (PR 13), only Fede's grades count (PR 14 home fallback, PR 13), 810 hand-off cases imported (PR 12); redeployed 17:2xZ; UI itself to be rebuilt per section 11 runs list, run detail, dataset and example pages; grading saved end to end on a local copy; no sideways scroll on a phone. Live site QA behind Access waits on the service token. |
| Bench runner (Benchmark PR 3) | Replays turns on the subscription through the door | merged |
| Bench imports (Benchmark PR 4) | Loads both benchmarks into the Benchmark as runs with your grades beside them | merged |
| Photo fix (PropFlow PR 22146) | Tenant photos in AppFolio text threads now reach Clara and the work order | merged |

Saturday Oct 25, five things, in order. Everything else below is Oct 24's list and is done.
Still local, no PR: the production prompt change for the new writer; the repair prompt's two rule lines (accepted version 3b). PR 22321 (post-call notes and stale held drafts block, never rewrite) merged Oct 25 afternoon; its three open choices are follow-up issues 22375 to 22377, the one worth a look: rejecting a stale held first email also drops the tour it booked (main already did this; 0 such cases in 14 days).
You answered all six tonight ("Finish everything"). Each row shows where it stands; this table is updated as the work lands.
| # | Your decision | Where it stands |
|---|---|---|
| 1 | Host the Benchmark like the ingest app: Cloudflare Pages behind Access at benchmark.propflowai.co, with its own table | live; 3 datasets, 9 runs, your 42 grades loaded One thing only you or Gera can do: create a Cloudflare Pages edit token in the dashboard and save it as the repo secret CLOUDFLARE_API_TOKEN, so pushes deploy by themselves. Until then deploys run from the Mac. |
| 2 | The Benchmark gets its own token instead of the shared robot token | live, proven 13:55Z: the door admits the Benchmark's own token and refuses the shared robot token (401). Email threads cannot yet be rebuilt through the door (issue 22181); recorded turns replay through PR 22179 once merged. |
| 3 | Recorded turns are kept 90 days, then deleted | done, expiry rule live |
| 4 | Drop the production rule that makes the first sentence "<home> is open now" | fix PR 22175 merged 15:3xZ, live everywhere on the next deploy replay of 72 real cards: "open now" openers 49 → 13, tour times offered 56 of 57 → 61 of 61; 9 of the 12 review cards same or better, the 3 worse ones were two one-off writer misses and one judge error (the address was right), no push needed |
| 5 | Ship the four mechanical first-reply fixes without the prompt split; the split stays on the Benchmark | merged 15:0xZ, live everywhere replay of 72 real cards: dashes 54 → 3, full size and rent line 17 → 24. Tour times slipped 56 of 57 → 48 of 52: when a first email had a dash, the one redraft that removes it dropped the tour times and the email was held unsent. Fix PR 22185 merged 16:0xZ: the redraft keeps every tour day and time; replay 62 of 62 offer times. Spanish Situs cards not replayable yet. |
| 6 | Work-order number: Clara tells the tenant the request is in and the number follows by text; one follow-up text with the PMS number when it lands | fix PR 22174 merged 15:5xZ, live everywhere; Willows live run proven 18:27Z Before: the confirmation quoted our internal id (PROP-000123 style). After, at the Willows (bench tenant TEST-204, real texts through Twilio): 18:27Z tenant texts "kitchen faucet is dripping"; 18:28Z confirmation says the work order number will come by text and quotes no internal id; 18:34Z the robot files it in AppFolio as #3319 and one follow-up text reads "Your work order number is 3319." Twilio shows exactly one. Evidence with message ids is on PR 22174. Read-only replay of the 14 tenant work orders Clara has filed: 8 get the number text, 1 raises the 24-hour alert, 5 have no phone on record. Willows maintenance texting was on "drafts" (set by another session Oct 23); set back to inherit 18:25Z for the run and left there. Two things seen outside this fix: the confirmation also repeated a bathroom-fan order from an earlier test that was marked done yesterday, and it contained em dashes. |
Not done tonight: runs through the real entry point; live proof of the photo fix (needs a real tenant photo); the "answer it, don't defer" fix is untested (the Benchmark lacks the customer's knowledge base).
Read-only inventory taken Oct 24 (code as of Oct 19, plus four paths added since). Nothing in production runs on Opus 5.5 today. Volumes are real customer traffic for Oct 18 to 25; our own test traffic is excluded.
| Group | Paths | Real weekly volume | Model today |
|---|---|---|---|
| Clara's main text and email reply | 1 (plus 3 pieces already moving) | ~120 real turns | Sonnet 5 |
| Voice line | 13 live agents | ~61 real calls | Sonnet 4.6 |
| Customer-facing writing | 19 | small, mostly under 20 a week each | Sonnet 4.6 / Haiku 4.5 |
| Decisions and classification | 59 | two biggest ~200 a week each | Haiku 4.5 |
| Internal writing | 15 | small | mixed |
The first three rows have a benchmark arm. The repair intake decision is measured end to end by the repair bench (did the right work order get filed), not replayed on its own. The two repair-chain rows that fire from the same tenant text but have no bench arm yet are in section 3.
| What it does for the customer | File : function | Model today, how chosen | Prompt, size | Weekly volume | Replay / eval | Audience |
|---|---|---|---|---|---|---|
| Clara's email reply to a prospect or tenant (main turn and tool rounds) | agents/clara/lib/agent/conversation-manager.ts : handleIncomingMessageTurn | claude-sonnet-5, REASON_FIRST_DRAFTING_MODEL via resolveDraftParams | clara-unified.ts getUnifiedSystemPrompt, about 640 lines, plus persona and email overlays | ~70 real runs (plus 18 from PropFlow staff); ~3,500 test runs; about 2.6 model calls per run | bench app (prospect-emails-v2), guest-card-regression skill, many evals | customer-facing |
| Clara's text reply to a tenant or prospect, including tenant repair texts | same file and function, SMS channel | claude-sonnet-5, same rule | same prompt, text overlay | ~53 real runs; ~400 test; ~195 pipeline lab | bench app (tenant-texts-v1, repair-episodes-v2), many evals | customer-facing |
| The first email Clara writes to a new lead, plus its redrafts | src/lib/domain/leasing/clara-guest-card-first-email-compose.ts : callModelWithReadOnlyTools | claude-sonnet-5, resolveDraftParams('email') | assembled from clara-unified.ts (2,001-line file) plus first-email overlay | ~25 real compose runs logged; 97 new lead records; 129 bench runs on Oct 25 | bench app (Bench A), guest-card-regression skill, first-email evals | customer-facing |
| Deciding which repairs in a tenant's text, email or call become work orders, and how urgent | src/lib/domain/maintenance/intake-classify.ts : classifyMaintenanceIntake | claude-sonnet-4-6, AGENT_MODEL | inline, about 60 lines | ~25 logged | eval scripts (maintenance-intake, emergency-definition, stated-unit); the repair bench measures its outcome end to end | customer-facing (decides the work order) |
| What it does for the customer | File : function | Model today, how chosen | Prompt, size | Weekly volume | Replay / eval | Audience |
|---|---|---|---|---|---|---|
| Clara's forced retry when she skipped a required lookup or described a tool instead of using it | conversation-manager.ts : handleIncomingMessageTurn (retry branches) | claude-sonnet-5 (email, text); sonnet-5-5 (video, staff desk); sonnet-4-6 (other channels) | same prompt as main turn plus a short correction | inside the main-turn call counts | covered with main turn | customer-facing |
| The one rewrite when a guard finds a wrong weekday or a leftover redaction marker | conversation-manager.ts : redraftReply | same as main turn | same prompt plus correction | inside main-turn counts | covered with main turn | customer-facing |
| Clara's quick booking-only reply on a thread a person already took over | conversation-manager.ts : runEscalatedTourBookingOnlyTurn | claude-sonnet-5 | buildTourActionOnlySystemPrompt, about 90 lines | unknown | none found | customer-facing |
| Clara's reply in video meetings | conversation-manager.ts, video channel | claude-sonnet-5-5, VIDEO_DRAFTING_MODEL | main prompt | none seen in 7 days | none found | customer-facing |
| Clara's reply on the staff leasing desk text line | conversation-manager.ts, staff desk | claude-sonnet-5-5, STAFF_DESK_DRAFTING_MODEL | main prompt | unknown | evals/lib/desk-thread.ts | internal staff, writes like customer copy |
| Clara's reply on the remaining channels (Telegram, any server-side voice turn) | conversation-manager.ts, MODEL = AGENT_MODEL | claude-sonnet-4-6 | main prompt | unknown, likely near zero | covered with main turn | customer-facing |
| The rewrite that removes "your tour is booked" when it is not | src/lib/domain/compliance/booking-promise-regenerate.ts : regenerateWithoutBookingPromise | claude-sonnet-5 | inline, about 20 lines | unknown (logs only on failure) | none found | customer-facing |
| The follow-up text or email Clara sends when a lead goes quiet | src/lib/domain/leasing/generate-followup-message.ts : generateFollowUpMessage | claude-haiku-4-5, FAST_MODEL | buildFollowUpPrompt, about 98 lines | ~8 logged generations | fast-tier bench (followup-copy-generator), eval (outreach-copy), replay (person-first composer) | customer-facing |
| A staff answer put into Clara's own words before it goes to the resident | src/lib/domain/escalation/relay-rephrase.ts : runRelayRephraseJudge | claude-haiku-4-5 | inline, about 14 lines | unknown | eval script (relay-own-words) | customer-facing |
| The question Clara texts a tenant on behalf of the repair tech | src/lib/temporal/activities/maintenance-comms.ts : composeViaHaiku (answerOrRelay) | claude-haiku-4-5 | inline, 1 to 2 lines | ~20 to 35 relays across the three rows | fast-tier bench (maestro-comms-compose, by name) | customer-facing |
| The note Clara passes back to the tech after the tenant answers | same file : composeViaHaiku (forwardAnswerToTech) | claude-haiku-4-5 | inline, 1 to 2 lines | see above | same | customer-facing (vendor) |
| The one-line status update Clara texts a tenant from the tech's message | same file : composeViaHaiku (composeStatusRelay) | claude-haiku-4-5 | inline, 1 to 2 lines | see above | same | customer-facing |
| Answering a vendor's question about their work order | src/lib/domain/vendors/handle-vendor-completion.ts : answerVendorQuestion | claude-haiku-4-5 | inline, about 4 lines plus work order | low (4 vendor-completion lines) | fast-tier bench (vendor-question-answerer) | customer-facing (vendor) |
| Spanish (or other language) arrival instructions inside a tour reminder | src/lib/i18n/translate.ts : translateOutboundFragment | claude-haiku-4-5 | inline, about 5 lines | unknown | fast-tier bench (outbound-translate) | customer-facing |
| The "around the home" lines (parks, shops, parking) for a listing | src/lib/domain/leasing/neighborhood-notes.ts : callNeighborhoodModel | claude-haiku-4-5, with web search | inline, about 58 lines | ~43 | none found | customer-facing (Clara quotes them) |
| The weekly portfolio briefing paragraph on the owner report | src/lib/domain/properties/portfolio-reports/presentation/clara-narrative.ts : generateClaraNarrative | claude-sonnet-4-6, AGENT_MODEL | inline, about 145 lines | unknown | none found | customer-facing (owner) |
| The "Improve with AI" button that tidies a manager's text blast | src/app/api/mass-sends/improve/route.ts : POST | claude-sonnet-4-6, AGENT_MODEL | mass-comms/improve.ts, about 19 lines | unknown (8 blast drafts this week) | none found | customer-facing |
| The question box on the public PropFlow website | src/app/api/public/faq-ask/route.ts : POST | claude-haiku-4-5 (route refuses non-Haiku) | faq-ask/prompt.ts, 74 lines | unknown | none found | public visitors |
| Rewriting a tenant's repair report into clean text for the tech | src/lib/domain/maintenance/dispatch-summary.ts : summarizeForDispatch | claude-sonnet-4-6, AGENT_MODEL | inline, about 21 lines | ~12 | eval script (maintenance-dispatch-title) | customer-facing (vendor) |
| What it does for the customer | File : function | Model today, how chosen | Prompt, size | Weekly volume | Replay / eval | Audience |
|---|---|---|---|---|---|---|
| Repair chain, not benchmarked yet: the last review of a drafted work order before it is filed (fix it or refuse it) | src/lib/maintenance/work-orders/judge.ts : judge | claude-haiku-4-5, FAST_MODEL | inline, about 45 lines | ~199 judge records (includes the daily maintenance test run) | fast-tier bench (wo-judge), evals/maintenance-judge-eval.ts, replay (work-order-pipeline-replay); not in the repair bench yet | internal (gates what is filed) |
| Repair chain, not benchmarked yet: reading when the tenant says the tech may enter and when they are home | src/lib/maintenance/work-orders/access-extractor.ts : extractAccessLogistics | claude-sonnet-4-6, AGENT_MODEL | inline, about 18 lines | ~100 | none found | internal |
| Labeling an incoming email (tour request, tour reply, question, maintenance) | agents/clara/lib/email/classify.ts : classifyWithAI | claude-haiku-4-5 | inline, about 45 lines | ~210 | fast-tier bench (email-category-classifier) | decides handling |
| Sorting an incoming email into prospect, operational or spam | agents/clara/lib/email/classify-spam-ai.ts : classifyEmailTriage | claude-haiku-4-5 | inline, about 54 lines | unknown (logs failures only; 5,060 email records incl. test) | fast-tier bench (email-triage-3way), replay (staff-mail-replay) | decides handling |
| Sorting an incoming text into tour request, tour reply or general | src/lib/domain/leasing/tour/classify-message.ts : classifySmsMessage | claude-haiku-4-5 | inline, about 10 lines | ~112 | fast-tier bench (sms-message-classifier) | decides handling |
| The fair housing check on every reply before it is sent | src/lib/domain/compliance/fair-housing.ts : runFairHousingJudge | claude-haiku-4-5 | inline, about 81 lines | ~70 to 140 | fast-tier bench (fair-housing-gate), eval scripts (gate, alert replay) | blocks sends |
| The fact-check of Clara's draft against what her lookups returned | agents/clara/lib/agent/guards/trajectory-judge.ts : runTrajectoryJudge | claude-haiku-4-5 | inline, about 30 lines | unknown | none found | blocks sends |
| Whether a new text is about a matter a person already took over | agents/clara/lib/agent/escalated-topic-classifier.ts : classifyTopicMatchWithRisk | claude-haiku-4-5 | inline, about 28 lines | unknown | replay scripts (mute-escalated-thread, escalated-topic-scope) | decides if Clara answers |
| Reading a reply to a tour or time-slot message (yes, cancel, reschedule, pick a slot) | src/lib/domain/leasing/scheduling/classify-intent.ts : classifyViaLLM | claude-haiku-4-5 | inline, about 44 lines | unknown | fast-tier bench (scheduling-intent-classifier) | decides tour action |
| Reading a prospect's reply to a tour offer by email | agents/clara/lib/email/extract-tour-reply-v2.ts : extractTourReplyV2 | claude-haiku-4-5 | inline, about 45 lines | none logged | fast-tier bench (tour-reply-extractor), replay (trace-replay) | decides tour action |
| Finding the tour a prospect asks for in their first email | agents/clara/lib/email/extract-tour-request-v2.ts : extractTourRequestV2 | claude-haiku-4-5 | inline, about 85 lines | ~6 | fast-tier bench (tour-request-extractor) | decides booking |
| Whether Clara offered one alternate tour time last message | agents/clara/lib/email/extract-clara-offer.ts : extractClaraOffer | claude-haiku-4-5 | inline, about 40 lines | none logged | fast-tier bench (clara-offer-extractor) | decides booking |
| Whether a lead is no longer interested, so follow-ups stop | src/lib/domain/leasing/prospect-summary.ts : classifyProspectDisqualification | claude-haiku-4-5 | inline, about 90 lines | unknown (within ~120 lead-summary runs) | fast-tier bench, eval script (disqualification) | stops outreach |
| Whether a quiet lead is waiting on us or said they would reach out | src/lib/domain/leasing/reengagement-intent.ts : classifyReengagementIntent | claude-haiku-4-5 | inline, about 25 lines | unknown | fast-tier bench (reengagement-intent-gate), dataset | decides a nudge |
| What Clara's drafted renewal text is about, so it can be held | src/lib/domain/leasing/renewal-orchestration/classify-reply-topic.ts : classifyReplyTopicVerdict | claude-haiku-4-5 | inline, about 40 lines | ~1 | eval script (reply-topic), promptfoo | holds sends |
| The team's reply to the monthly renewal list: approve, hold, change rent, skip | src/lib/domain/leasing/renewal-decisions/engine.ts : decideRenewals | claude-sonnet-5-5, DECISION_ENGINE_MODEL | inline, about 36 lines | ~10 (monthly batch) | eval script (renewal-decisions) | decides offers |
| The legal check on a rent-collection message | src/lib/domain/compliance/collections/legal-review.ts : reviewCollectionsMessage | claude-haiku-4-5 | inline, about 64 lines | ~1 | fast-tier bench (collections-legal-gate) | blocks sends |
| Checking a building-wide text for private details before it goes out | src/lib/domain/messaging/mass-comms/broadcast-safety.ts : classifyBroadcastLeak | claude-haiku-4-5 | inline, about 28 lines | up to ~8 | fast-tier bench (broadcast-leak-gate), eval script | blocks a blast |
| The manager's yes/no reply to a mass-text confirmation | src/lib/domain/mass-comms/handle-mass-comms-confirm-reply.ts : decideMassCommsReply | claude-haiku-4-5 (exact words answered first) | confirm-reply-prompt.ts, about 55 lines | up to ~8 | fast-tier bench (mass-comms-confirm-reply) | sends a blast |
| A manager's reply about an unknown caller claiming to be a tenant | src/lib/domain/tenant-confirmation/handle-confirmation-review-reply.ts : decideConfirmationReviewReply | claude-haiku-4-5 | inline, about 10 lines | up to ~32 | fast-tier bench (pm-confirmation-review-reply) | grants tenant access |
| Whether a finished call asked to stop texts or calls | src/lib/domain/compliance/voice-postcall-opt-out-sweep.ts : classifyOptOutTranscript | claude-sonnet-4-6, AGENT_MODEL | inline, about 8 lines | unknown (once per finished call, ~61 real calls) | none found | stops contact |
| After a call: is the caller a new renter lead, and what do they want | src/lib/integrations/voice/postcall-prospect-capture.ts : extractProspectWithOutcome | claude-haiku-4-5 | inline, about 50 lines | ~46 | fast-tier bench, replay (postcall-capture, spoken-home-calls) | creates the lead |
| After a call: fill missing name, bedrooms, move-in date on the guest card | src/lib/integrations/voice/postcall-guest-card-reconcile.ts : reconcileGuestCardFromCall | claude-haiku-4-5 | inline, about 40 lines | unknown | fast-tier bench (postcall-guest-card-reconcile) | internal record |
| After a call: spot a caller who says they live in a unit | src/lib/integrations/voice/postcall-claimed-unit-capture.ts : maybeCaptureClaimedUnit | claude-haiku-4-5 | inline, about 50 lines | ~44 | fast-tier bench | internal (manager approves) |
| After a call: pick up an unknown caller's name | src/lib/integrations/voice/postcall-unknown-caller-name.ts : maybeCaptureUnknownCallerName | claude-haiku-4-5 | inline, about 47 lines | ~15 | fast-tier bench | internal record |
| After a hand-off call: pull tour bookings and callback promises out of the call | agents/clara/lib/voice/post-transfer.ts : extractActionsFromTranscript | claude-haiku-4-5 | inline, about 45 lines | unknown (up to ~61 real calls) | fast-tier bench (post-transfer-extractor), replay scripts | customer-facing (books follow-ups) |
| Staff email reply: find promises Clara must follow up on | agents/clara/lib/email/extract-pm-commitments.ts : classifyCommitments | claude-haiku-4-5 | inline, about 48 lines | unknown | fast-tier bench (pm-commitment-extractor) | internal follow-ups |
| Pull issue type, urgency, renewal intent from a tenant or owner email | agents/clara/lib/email/extract-operational-signal.ts : extractOperationalSignal | claude-haiku-4-5 | inline, about 81 lines | ~66 | fast-tier bench (operational-signal-extractor) | internal views |
| Reading a scanned PDF attachment | agents/clara/lib/email/extract-attachment-text.ts : ocrPdf | claude-haiku-4-5 | inline, about 14 lines | unknown (~21 inbound attachments) | none found | internal input |
| Tagging each message in a thread with its topic | src/lib/domain/conversations/message-topic-classifier.ts : classifyThreadTopics | claude-haiku-4-5 | inline, about 100 lines | ~60 to 170 | labeler benchmark, mutation control | internal |
| What to remember from a staff reply (company rule, property fact, nothing) | src/lib/domain/escalation/teaching-capture.ts : classifyAndDistillTeamAnswer | claude-sonnet-4-6, AGENT_MODEL | inline, about 41 lines | unknown | replay scripts (staff-reply-intake, team-answer) | internal knowledge |
| Listing the promises Clara made in a conversation | src/lib/domain/promises/extract-promises.ts : extractPromises | claude-sonnet-4-6, AGENT_MODEL | inline, about 69 lines | unknown (74 promise records) | replay (promises scenario, goldmine) | internal |
| Vendor email: was a visit agreed, and when | src/lib/domain/vendors/email-schedule-extraction.ts : classifyVendorEmailSchedule | claude-haiku-4-5 | inline, about 100 lines incl. tool | unknown (139 vendor visit records) | eval scripts (vendor-email golden, schedule) | books the visit |
| Is this email from a vendor we already know under a new name | src/lib/domain/vendors/stated-identity-adjudication.ts : adjudicateStatedIdentity | claude-haiku-4-5 | inline, about 42 lines | unknown | eval script (vendor-email golden) | internal |
| What a vendor's text is: question, done, times, update | src/lib/domain/vendors/handle-vendor-completion.ts : classifyVendorMessage | claude-haiku-4-5 | inline, about 64 lines | low | fast-tier bench (vendor-message-intent-classifier) | decides relay |
| Turning a vendor's availability message into time slots | same file : parseVendorSlots | claude-haiku-4-5 | inline, about 2 lines | low | fast-tier bench (vendor-slot-parser) | slots relayed to tenant |
| Whether a vendor photo is a real completion photo | same file : isValidCompletionPhoto | claude-haiku-4-5 (vision) | inline, about 2 lines | low | fast-tier bench (vendor-completion-photo-validator) | internal |
| Vendor phone call: was a visit booked, declined or moved | src/lib/domain/vendors/postcall-extraction.ts : classifyVendorCallTranscript | claude-haiku-4-5 | inline, about 100 lines incl. tool | 0 (vendor calling agent had no calls) | fast-tier bench (vendor-call-outcome-extractor) | books the visit |
| Why the vendor is visiting (second call on the same vendor call) | same file : classifyVendorVisitPurpose | claude-haiku-4-5 | inline, about 16 lines | 0 | eval script (vendor-call-purpose) | internal |
| Who pays for a move-out item when the rules cannot tell (3-run panel) | src/lib/domain/turnover/condition-attribution-judge.ts : callJudge | claude-sonnet-4-6, AGENT_MODEL | inline, about 19 lines | unknown | none found | internal (manager reviews) |
| Reading move-out notes and photos into room-by-room condition | src/lib/domain/turnover/condition-extraction.ts : runExtractionOnce | claude-sonnet-5, EXTRACTION_MODEL | inline, about 40 lines | unknown (39 condition reports created) | model comparison eval only | internal |
| Flagging odd move-out charge wording | src/lib/domain/turnover/judge-turnover-charges.ts : judgeTurnoverCharges | claude-haiku-4-5 | inline, about 10 lines | unknown | fast-tier bench (turnover-charge-judge), turnover replay | internal |
| Reading the current rent specials off each community website, nightly | src/lib/platform/scrapers/shared.ts : extractWithClaude (concession watcher) | claude-haiku-4-5 | inline, about 12 lines | nightly, one call per tracked site | fast-tier bench (website-scraper-extractor) | feeds what Clara quotes |
| Operator-triggered website read (amenities, specials, hours) | same function via scrapeAndSave | claude-haiku-4-5 | inline, about 44 lines | rare | same bench | internal |
| Extracting a prospect's name from a call or chat | src/lib/domain/leasing/prospect-summary.ts : extractNameFromTranscript | claude-haiku-4-5 | inline, about 45 lines | unknown | fast-tier bench (prospect-name-extractor) | internal record |
| Whether a new policy option repeats an existing one | src/lib/domain/policies/core/option-similarity.ts : findSimilarOption | claude-haiku-4-5 | inline, about 9 lines | rare | none found | internal |
| Short property code for a newly synced property | src/lib/domain/properties/generate-ticker.ts : generateTicker | claude-haiku-4-5 | inline, about 40 lines | ~0 | fast-tier bench (property-ticker-generator) | internal |
| Learning the layout of an uploaded weekly report | src/lib/domain/properties/portfolio-reports/learn/learn-config.ts : callLearnModel | claude-opus-4-8, LEARN_MODEL | inline, about 23 lines | rare (on upload) | none found | internal |
| Learning the column layout of a client's rollup template | src/lib/domain/properties/portfolio-reports/output/learn-output-config.ts : callLearnOutputModel | claude-sonnet-4-6, AGENT_MODEL | inline, about 14 lines | rare | none found | internal |
| Rent roll spreadsheet with an unknown layout: which column is what | src/lib/domain/leasing/rent-roll/parsers/llm-fallback.ts : identifyColumnsWithLlm | claude-haiku-4-5 | inline, about 55 lines incl. tool | rare | fast-tier bench (rent-roll-llm-fallback) | internal |
| Rent roll PDF into units, tenants, rents, dates | src/lib/domain/leasing/rent-roll/parsers/pdf-vision.ts : parsePdfWithVision | claude-haiku-4-5 | inline, about 35 lines | rare | fast-tier bench (rent-roll-pdf-vision) | internal |
| Yardi property activity report PDF into numbers | src/lib/domain/properties/reports/par-parser-vision.ts : parsePARReportVision | claude-haiku-4-5 | inline, about 193 lines | 0 | fast-tier bench, benchmark script | internal |
| Property rules and facts out of an uploaded document (text, then scanned) | src/lib/platform/parsers/knowledge-extractor.ts : callHaikuText, callHaikuVision | claude-haiku-4-5 | inline, about 36 lines | rare (on upload) | fast-tier bench (knowledge-extractor) | internal knowledge |
| Maintenance manual PDF into equipment data | src/lib/platform/parsers/maintenance-manual-parser.ts : callHaiku | claude-haiku-4-5 | inline, about 44 lines | rare | fast-tier bench | internal |
| The setup chat for a new property from a rent roll | src/app/api/properties/setup-chat/route.ts : POST | claude-haiku-4-5 | inline, about 61 lines | unknown | fast-tier bench (property-setup-chat) | staff |
| Dormant: first look at a batch of tenant messages (Maestro) | src/lib/domain/maestro/precheck.ts : classifyPackagePrecheck | claude-haiku-4-5 | inline, about 14 lines | 0 (no production start found) | fast-tier bench (maestro-precheck-router) | would be customer-facing |
| Dormant: deciding Clara's actions from a batch of tenant messages, 3 runs (Maestro) | src/lib/domain/maestro/reasoning-brain.ts : reasonOnce | claude-sonnet-4-6, pinned | inline, about 45 lines | 0 | none found | would be customer-facing |
| Not wired: ask staff or hand off when Clara is stuck | src/lib/domain/escalation/staff-loop/judgment.ts : judgeStuck | claude-sonnet-5-5 | inline, about 17 lines | 0 | benchmark script, real-turns dataset | internal |
| Not wired: is a staff note on a guest card safe for Clara to see | src/lib/domain/leasing/guest-card-note-model-stage.ts : classifyGuestCardNoteByModel | claude-haiku-4-5 | inline, about 37 lines | 0 | none found | internal |
| What it does | File : function | Model today, how chosen | Prompt, size | Weekly volume | Replay / eval | Audience |
|---|---|---|---|---|---|---|
| The short note on a lead's card (what they want, budget, lease length) | src/lib/domain/leasing/prospect-summary.ts : generateProspectSummary | claude-haiku-4-5 | inline, about 125 lines | ~120 | fast-tier bench (prospect-summary-generator) | staff |
| The one-sentence staff notice: what a caller, texter or emailer wants | src/lib/team-notify/summarize.ts : summarize | claude-sonnet-5, SUMMARY_MODEL | inline, about 50 lines incl. grounding check | unknown | replay (post-call-notice-replay, team-notify-replay) | staff (client's team) |
| Facts from a finished call for the staff call-back notice | src/lib/team-notify/voice-handoff.ts : liveVoiceFacts | claude-sonnet-5, SUMMARY_MODEL | voice-facts.ts, about 34 lines | unknown (up to ~61 real calls) | replay (post-call-notice-replay) | staff |
| Staff summary of an incoming email and attachments | agents/clara/lib/email/summarize.ts : summarizeEmail | claude-haiku-4-5 | inline, about 8 lines | ~8 | fast-tier bench (email-summarizer) | staff |
| The one-line gist on the operations feed card after a call | src/lib/domain/ops-feed/ops-gist.ts : gistForOpsCard | claude-haiku-4-5 | inline, about 19 lines | ~7 to 60 | replay (replay-ops-gist) | staff |
| Turning a staff reply into a clean policy note in the knowledge base | src/lib/domain/escalation/teaching-capture.ts : distillStaffAnswer | claude-sonnet-4-6, AGENT_MODEL | inline, about 14 lines | unknown | replay scripts (teach-distill) | staff, later quoted by Clara |
| What was wrong, fixed and replaced, from a vendor's completion message | src/lib/domain/vendors/parse-completion-notes.ts : parseCompletionNotes | claude-haiku-4-5 | completion-notes-prompt.ts, about 4 lines | low | fast-tier bench (vendor-completion-notes-parser) | staff |
| Answering a manager's question about an unverified tenant claim | src/lib/domain/tenant-confirmation/handle-confirmation-review-reply.ts : answerClaimQuestion | claude-haiku-4-5 | inline, about 7 lines | rare | none found | staff |
| Summary of a team video meeting above the transcript | src/lib/domain/video-clara/meeting-summary.ts : summarizeVideoCall | claude-sonnet-4-6, AGENT_MODEL | inline, about 19 lines | 0 | none found | staff |
| Short tenant summary on the tenant page | src/app/api/tenants/[id]/summary/route.ts : generateTenantSummary | claude-haiku-4-5 | inline, 1 long line plus facts | ~11 | fast-tier bench (tenant-summary-generator) | staff |
| Insights on a property report page | src/app/api/property-reports/insights/route.ts : POST | claude-haiku-4-5 | inline builders, size not counted | unknown | fast-tier bench (portfolio-insights-generator, by name) | staff |
| Rough repair price range on a work order | src/app/api/work-orders/[displayId]/market-estimate/route.ts : GET | claude-haiku-4-5 | inline, about 18 lines | unknown | fast-tier bench (wo-market-estimate) | staff |
| Ask Clara, the dashboard chat for owners and managers | src/app/api/dashboard/chat/route.ts : streamTurn | claude-sonnet-5, CHAT_AGENT_MODEL | dashboard/chat/system-prompt.ts, 117 lines plus tools | unknown | promptfoo (dashboard-chat, dashboard-chat-voice) | customer's staff |
| Grading a finished Clara conversation for a red alert | src/lib/domain/conversations/conversation-judge.ts : runConversationJudge | claude-haiku-4-5 | inline, about 44 lines | ~180 runs, mostly grading the test mirror | calibration script, grading corpus | internal QA |
| After-the-fact fair housing grade of sent replies | src/lib/domain/compliance/fair-housing.ts : runFairHousingJudgeWithCitation | claude-haiku-4-5 | inline, about 77 lines | part of 1,154 grade records, mostly test | eval script (citation) | internal QA |
All live agents run claude-sonnet-4-6, set on the agent in ElevenLabs (repo source: agents/clara/lib/voice-agents/*.config.json and coworker.ts). ElevenLabs offers claude-opus-5-5. Callers reach the specialist agents by a transfer from Triage, so calls are counted under Triage. Prompt size is in characters; divide by 4 for approximate tokens.
| Agent | What it does for the caller | Model | Prompt size | Calls, 7 days | Replay / eval |
|---|---|---|---|---|---|
| Clara Triage (production) | answers every call, routes to the right specialist | claude-sonnet-4-6 | 90,061 chars (~22.5k tokens) | 59 real | voice harness, fastlane skill, ElevenLabs evaluation criteria |
| Clara Leasing | prospects: homes, prices, tours | claude-sonnet-4-6 | 143,472 chars (~36k tokens) | via Triage | voice harness, leasing evals |
| Clara Maintenance | tenant repairs, emergencies | claude-sonnet-4-6 | 69,131 chars (~17k tokens) | via Triage | voice harness, emergency evals |
| Clara Resident Services | lease, billing, general tenant questions | claude-sonnet-4-6 | 32,320 chars (~8k tokens) | via Triage | promptfoo (tenant-resident-services) |
| Clara Unknown Caller | callers the line cannot identify | claude-sonnet-4-6 | 43,542 chars (~11k tokens) | via Triage | single-building-unknown-caller eval |
| Clara Renewal (inbound) | renewal questions | claude-sonnet-4-6 | 41,339 chars (~10k tokens) | via Triage | renewal evals |
| Clara Maintenance (Handyman) | the in-house tech calling in | claude-sonnet-4-6 | 33,277 chars (~8k tokens) | via Triage | voice harness |
| Clara Turnover Intake | move-out inspection intake | claude-sonnet-4-6 | 14,860 chars (~4k tokens) | via Triage | none found |
| Clara Emergency Relay (outbound) | calls on-call staff about an emergency | claude-sonnet-4-6 | 2,799 chars | 2 real | emergency evals |
| Clara Renewal (outbound) | calls tenants about renewal | claude-sonnet-4-6 | 33,683 chars | 0 | renewal evals |
| Clara Vendor Calling (outbound) | calls vendors to schedule visits | claude-sonnet-4-6 | 79,405 chars | 0 | vendor demo harness |
| Clara Escalation Answer Callback (outbound) | calls back with a staff answer | claude-sonnet-4-6 | 8,696 chars | 0 | none found |
| Clara Coworker Line | the staff phone line for leads, tours, work orders | claude-sonnet-4-6 (coworker.ts LLM constant) | 4,932 chars | 0 | coworker harness |
| Willows copies of all the above | test line | claude-sonnet-4-6 | same | 40 test (Triage) | n/a |
| Voice Eval Robot, Robo-Vendor, demos, prototypes | test callers | claude-sonnet-4-6 (one prototype on qwen) | small | 81 test | n/a |
| What it does | File : function | Model |
|---|---|---|
| The note Clara writes on a lead's guest card after each outcome, plus its independent fact check | src/lib/domain/leasing/card-notes/card-note-writer.ts : writeCardNote (via compose-card-note-live.ts) | claude-opus-5, CARD_NOTE_MODEL (Opus 5, not 5.5) |
| One retry of the first-email writer when its check fails, then hold for a person | agents/clara/lib/agent/writer-call-capture.ts : retryWriterOnce | same model as the writer (claude-sonnet-5) |
| First-email writer moved to a shared read-only draft helper | src/lib/domain/leasing/read-only-leasing-draft.ts : callModelWithReadOnlyTools | claude-sonnet-5, resolveDraftParams('email') |
| A second, emergency-only read of a repair report when intake said "not an emergency" | src/lib/domain/maintenance/emergency-content-check.ts : checkEmergencyContent | claude-sonnet-4-6, AGENT_MODEL |
| Name in code | Model today | Can an env var change it? | Set in production? |
|---|---|---|---|
| FAST_MODEL | claude-haiku-4-5 | no | n/a |
| AGENT_MODEL | claude-sonnet-4-6 | CLAUDE_MODEL | not set on Vercel or the worker, so Sonnet 4.6 everywhere |
| REASON_FIRST_DRAFTING_MODEL (email and text replies, first email) | claude-sonnet-5 | no | n/a |
| CHAT_AGENT_MODEL (Ask Clara) | claude-sonnet-5 | no | n/a |
| VIDEO_DRAFTING_MODEL, STAFF_DESK_DRAFTING_MODEL | claude-sonnet-5-5 | no | n/a |
| EXTRACTION_MODEL (move-out inspection) | claude-sonnet-5 | CONDITION_EXTRACTION_MODEL | not set |
| SUMMARY_MODEL (staff notices) | claude-sonnet-5 | no | n/a |
| DECISION_ENGINE_MODEL, JUDGE_STUCK_MODEL | claude-sonnet-5-5 | no | n/a |
| LEARN_MODEL (portfolio report layout) | claude-opus-4-8 | PI_LEARN_MODEL | not set |
| Maestro models | Haiku 4.5 / Sonnet 4.6 | MAESTRO_PRECHECK_MODEL, MAESTRO_REASONING_MODEL | not set |
| ElevenLabs voice agents | claude-sonnet-4-6 (set on each agent in ElevenLabs) | no | ElevenLabs lists claude-opus-5-5 as available |
Nothing in production runs on Opus 5.5 today. The one Opus row at HEAD is the report-layout learner (Opus 4.8); newer main adds guest-card notes on Opus 5 (not 5.5).
How to read the volume column: "real" means customer properties and customer senders. "Test" means Willows, Fairhaven, the pipeline lab, and PropFlow's own test mailbox. That mailbox ran about 3,500 replayed emails onto mirror copies of customer properties this week, 2,072 of them on Oct 24. "Unknown" means no per-call log or record exists to count. Coverage labels: replay = a script that replays real production records; fast-tier bench = a scored set under evals/fast-tier/ (built from a real corpus during the Haiku tier work); eval = promptfoo or scenario evals; bench app = a clara-benchmark dataset; none.
src/, agents/ and the Temporal activities that calls the Anthropic SDK or names a model. Tests, evals, harnesses and tooling scripts are excluded. No production entry point into scripts/ was found (two Lambda handlers mention the folder and were not opened, so this is not exhaustively verified). No model calls were found in the AppFolio robot's (mock-pms-service) TypeScript.AgentTrace, split by property and sender to separate test traffic), CloudWatch Logs Insights over the Vercel runtime and worker log groups, and the ElevenLabs conversation list. Many paths log only on failure, so they show as unknown. Staff-page routes could not be counted because their log marker matched a route-listing line.Independent final pass, Fable 5.1 at max effort, Oct 25 evening, over the published page (before the Cerebrus section was added). Findings are the reviewer's; nothing on the page was changed in response yet. Where the two reviews agree, working separately: the "I've told the team" reply is still permitted in several places; the page asks Fede far more than three things; the Benchmark ledgers and the body disagree on whether the new first-email writer has a switch. Opus alone leads with "nothing measures noise"; Fable alone leads with the asks cut. One finding in both reviews is withdrawn: the claim that a Slack approval card on tenant replies contradicts "never a queue". Withdrawn (Fede, Oct 25): "I never said I didn't want an approval queue." His Sept 8 autonomous rule covers identity and household decisions, not a person reviewing Clara's dark tenant replies. The approval card stays as designed.
Not yet: the page states the right objective at the top and then contradicts it in the design, and it asks Fede to choose between options that disagree with each other. First, pick one answer on supervised mode: the Supervised section and Figure 13 say the customer's own Send button is the only hold left, while the Tenant communications diagram and the Friday row make every tenant reply wait on a Slack card until a person approves it. [Withdrawn as a contradiction: Fede keeps the approval card; the fix is to make the Supervised section say so.] Second, strike the four places that still let Clara say "I'll check with the team" and send one more notice: rollout step 2's tracked-ask exception, decision 5's option (b), the repair writer in prompt 3b, which alerts the team on its own judgment with no emergency gate, and the new first-email writer, which puts answers off twice as often as today's. Third, cut the asks to the three that are Fede's (A1 to A3) and mark the rest decided or recorded; today the page carries more than thirty inputs for him, including a switch question his Sep 30 rule already answers. With those three edits, and the untraceable "intro texts proven at the Willows" sentence removed, the page is ready to decide from; Tuesday and Saturday are the only days a builder could start without asking, and Day 0 only for the first-email surface.
Every interface, habit, outbound and reply pattern on the page, read against the one objective. Noisier items first, in order of severity; the quieter items at the end are the parts worth keeping as written.
| Item (section) | Quieter or noisier | Why | Change proposed |
|---|---|---|---|
| Hold card in property Slack, "every reply stops at the hold card until a person approves" (Tenant communications, Today's path vs proposed; Fri Oct 31) | Noisier | One Slack card per tenant reply is a per-message team item; 95 of 142 and 101 of 120 held drafts expired unseen, and Fede ruled on Oct 21 that hold mode is the problem. | Replace with what the Supervised section already says: shadow-and-diff where staff answer in AppFolio, live answers on covered topics on Clara's own line, silence plus a tracked ask for the rest. Delete the card from the diagram. |
| "No 'with the team' without a tracked ask" (Proof and rollout, step 2) | Noisier | The Western Slope anti-pattern had a tracked ask (an email to the team) and was still worse than silence; the exception keeps both the sentence and the notice alive. | Rewrite the fix: the sentence is never written. When Clara cannot help she is silent, the ask is recorded, and the owner hears about it once, in the morning list. |
| The 3b repair writer alerts the team on its own judgment, with no emergency gate (Ledgers, filing table, Situs 1159587096; Four repair cases, case 4; shipped Thu Oct 30) | Noisier | With both halves on Opus the writer alerted the team on a trash complaint and told the tenant "ya alerté al equipo" (the page says this breaks the no-team-items rule); in the accepted arm, case 4 told Karin the team was alerted and it was urgent, after a wrong attach. Thursday ships that writer with no gate named. | Add a blocking check before Thursday: a team alert leaves only when the emergency content check says emergency; otherwise the writer files, asks the unit, or stays silent. |
| New first-email writer puts answers off 6 / 5 / 6 against today's 3 / 3 / 4 ("I'll check with the team", pairs 6, 12, 14, 15) (Your review set, the numbers) | Noisier | The new writer defers twice as often as today's, and the release plan's exit test ("worse on a fact, a hold or tone") does not count a put-off as worse. | Make "no put-off when the fact is on file" a blocking check and a stage 1 exit criterion before any company moves to the new writer. |
| Decision 5, option (b) "short code-built reply + team item", kept as "a reviewer prefers (b)" (Decisions for you) | Noisier | Option (b) is the per-message team item Fede removed on Oct 23, plus a reply that tells the renter nothing. | Strike (b). After a failed retry the turn is silent and one ask goes to the owner with the draft attached; the renter gets no filler. |
| Daily re-ask on the ask record, the nag cron (Where it stands, "a daily re-ask"; Week one, "she never nags and never redoes") | Noisier | A live automatic staff message with no setting named anywhere on the page, while Week one promises she never nags. | Fold the nag into the morning list under the same company setting; until then the page must name it as the one reminder running in production today. |
| Email to the property manager per hand-off and "I'll check with the team" to the tenant (Tenant communications, TODAY diagram: "escalation row only if the property names an owner email") | Noisier | Today's code sends one email per hand-off and a filler reply; step 2 removes neither explicitly. | List both under step 2 as removed: one notice per matter through askTheTeam, then the morning list; the tenant hears nothing until there is an answer. |
| Review-mode hold, "102 fired, 1 real catch: Keep" (Every gate between Clara and a send) | Noisier | 101 staff interruptions for one catch, and Figure 13 row 14 deletes the same hold two sections later. | Change the action to "delete with row 14"; the one real catch becomes a blocking check that needs no person. |
| Hold on maintenance texts, "120 held, 101 expired unsent: Fix who works it" (Every gate) | Noisier | "Who works it" rebuilds a queue; 101 renters in 14 days waited for a person who never came. | Replace with shouldSpeak: answer covered repairs live and file the order, silence plus an ask for the rest; no queue to work. |
| "Held drafts that expire in silence become an approval that stays open" (The seven days, Fri Oct 31) | Noisier | An approval that stays open is the queue the Supervised section rejects; it moves the silence from the renter to a staff backlog. | Make it an ask with an owner that shows on the morning list, never an open approval. |
| Intro texts and the hello per role, running "beside the refactor from day 1" (Advice on the order; Week one) | Noisier at first | An unasked text to each staffer is new outbound; the old page says no customer has asked for it and nothing has run. | Keep it behind the setting. The Situs kickoff call asks who wants texts (A2) before any hello leaves; the first intro is the human's, not Clara's. |
| "Since Monday I've answered 14 renters... Try me: text 'what's on my calendar tomorrow'" (Week one, How we get them excited) | Noisier | A second message after the invite, in marketing copy, into a staff inbox. | Put the sentence inside the invite body; never a separate text. |
| Step 4 new outbound: "Clara chimes in inside their shared inbox" and "Clara proposes PMS settings" (Proof and rollout; Figure 12) | Noisier | Habit D is not built and no customer asked; a chime-in is "one more email" by construction, and unprompted settings advice is a notice nobody requested. | Remove both from step 4 until a customer asks; keep PMS advice on the dark register only, with no schedule. |
| "Transfer confirmed by text to the caller" (Proof and rollout, step 4) | Noisier | A text after every transfer, on a page that puts phone out of scope. | Send only when the transfer had no human leg (70 and 61 such transfers in the audit), as the checked callback in Figure 13 row 10. |
| The 8:01 AM text to David five hours after his first email ("I sent you an email about your inquiry. If texting is easier...") (Ledgers, your morning list) | Noisier | A second-channel ping with no new information, and the ask to Fede about it is buried in a bullet of the ledger. | Rule: that text goes within minutes only when the renter asked for a call or a text and left a number; otherwise never. Move the ask to Decisions or close it. |
| Staff notices, about 20 templates, "keep: one notice engine" (Figure 13, row 13) | Neutral to noisier | Twenty templates survive with no count target and no setting. | Each template is either an ask (folds into the record) or a tracked notice under a company setting; the page lists which are deleted. |
| Shadow cards with rating buttons in Slack (Ledgers, every ask, Oct 25 row) | Neutral if the channel is ours | One card per tenant text is right for grading, but the page does not say which channel and the row was added to a ledger labeled verbatim. | State "our channel, never the customer's", and that the cards stop once the Clara Bench page holds the labels. |
| Emergency: "reports to the team at once before the unit is known", decided under the grant (Ledgers, done means, row 3) | Quieter when real, noisier when wrong | A customer-facing behavior decided without Fede, and its first replay alerted the team on a trash complaint. | Move it into Decisions for Fede with the emergency definition attached, and gate it on the emergency content check. |
| "Did a person reply?" trained on AppFolio; identity checked before a policy answer (The idea; step 2) | Quieter | Both stop a reply before it is written, with no message to anyone. | Keep. Name the replay that proves each. |
| shouldSpeak's four questions (The idea) | Quieter | One decision, one reason logged, and silence is the answer to three of the four questions. | Keep. Give question 4 its test: the reply tells the tenant something new that is on file; otherwise silent. |
| askTheTeam, one notice per matter to one owner (Four interfaces) | Quieter | Thirteen (or seventeen) paging paths become one notice, and the morning list replaces an email per item. | Keep. Add "no notice when askState is reached": the owner already on the thread is never paged. |
| askState, "never 'it's with the team' unless the record says so" (Four interfaces) | Quieter | Removes the three-cats failure class at the sentence level. | Keep, and make the blocked sentence a silent turn, not a reworded promise. |
| Staff-touch check before every send and the free code gate before any model (Figure 13 row 15; Tenant communications) | Quieter | "Is a person already on it" is decided before a model runs; the staff-note fix of Oct 24 shows it working on real threads. | Keep. It is question 3 of shouldSpeak and should be named as such. |
| Decomposed repair prompt on Opus 5.5: "check with the team" 34% to 8%; "stayed out when the office was running the thread" 0 to 8 of 10; one order for the two-laundry-room text (The bench; Ledgers, full mode) | Quieter | The quietest arm in the benchmark is also the most helpful one. | Keep. Count "clear repair report left silent" (0 to 1 of 5) and the Spanish misses as the under-reply side of the same measure, so quiet never becomes mute. |
| Habit E, take work and report on the same thread (Live as a coworker; Figure 13 row 3) | Quieter | Work arrives from staff where they already are and the report goes back on that thread; no new thread, no notice. | Keep. The "one confirm rule" means at most one question, and only when the ask is ambiguous. |
| PmsInbox adapter: read texts, read staff replies, post a text (Four interfaces) | Quieter | Reading what staff already said in the customer's system answers "is a person already on it" without asking anyone; the mirror exists at Western Slope. | Keep. Name Situs (no mailbox copy) as the gap that makes shadow agreement unmeasurable there until the adapter reads its inbox. |
| Morning list as the only reminder, a company setting, default off (The idea; A3) | Quieter | One message a day replaces an email per item, and the digest cron already exists (built Oct 23, off). | Keep. A3 should say the cron is reused and the nag retires into it. |
Where the "I couldn't help, I've told the team" reply still survives on the page:
Automatic staff messages on the page that are not a tracked ask:
Reminders or digests without a setting: only the daily re-ask cron. The morning list and the resident-questions digest both carry a setting (default off) wherever they appear; the page should say they are the same cron.
Line numbers below refer to the tag-stripped text of the page and of the two source extracts, not to the HTML. Labels: HARD (traced to a source line), ESTIMATE (a number with no derivation), INVENTED (no trace in any file), UNVERIFIABLE (cannot be checked from the extracts), CONTRADICTED (two statements disagree); "inconsistent" means one quantity with several values on the page. Ranked by how much each changes a decision.
| Claim (section, page line) | Source | Label | What to do |
|---|---|---|---|
| "the intro texts, proven at the Willows on Oct 25 from the building line" (Advice on the order, 569) | No trace in either extract; the old page says the intro has no customer ask yet (152) and that nothing has run on Willows self-onboarding (185). | INVENTED (production logs were not read for this review) | Cite a message id and the time, or drop "proven". |
| "Supervised = every reply stops at the hold card until a person approves" (Tenant communications diagram, 488) against "The customer's own Send button is the only hold left" (Supervised, 187) and Figure 13 row 14 "delete: no hold on Clara's own channels" | Benchmark page 1054 (hold every tenant reply, no bypass in week one) against the old page's round-2 ruling. | CONTRADICTED on the page | Pick one. The Supervised ruling is the newer and matches Fede's Oct 21 words. |
| "The Benchmark lane's own seven-day plan is this one." (Order of work, 601) | Benchmark 0e (456 to 475) is a different week: Day 4 the bugs the bench found, Day 5 the first email to production behind a setting, Day 6 the when-to-reply table, Day 7 the voice line, plus a "What Fede does this week" list. None of these is in the merged table. | CONTRADICTED | Say "supersedes" and list what was dropped, or restore the dropped days. |
| Release plan "ships behind a company setting (default: today's writer)" (0c, 677) and the Dark line (536) against done-means row 1 "the production PR (decomposed first email on Opus 5.5, no switch)" (Ledgers, 1478) and the cheap-vs-big sentence "a fix, no switch" | Benchmark 202 and 248 carry both. | CONTRADICTED; Fede's Sep 30 rule already decides it (an improvement to live behavior ships with no switch) | Remove "switch or no switch" from the morning list; stage 2 becomes "live everywhere after the shadow exit". |
| "decisions for Fede holds three questions" (Where the reviewers disagreed, 1258); "Three questions is the cap Fede set for this page" (old 600) | The page carries A1 to A3, Benchmark 1 to 6, release plan (1) and (2), tenant communications (1) to (4), 15 marks, 4 repair verdicts, three un-numbered asks in the morning list, and a second, already answered set of six decisions that shares the numbers 1 to 6. | CONTRADICTED | Three asked. Everything else labeled "recorded, not asked" or closed under a standing rule. |
| Review set: "Your marks decide whether new Clara goes live", three frozen runs in the same hour (Oct 24 night, 697 to 722), against done-means row 1 ending at "Parity bar NOT met ... the exact ten-line prompt has run once" (rerun 4, Oct 25 about 03:00Z, 1478) and 0b "worse than production on 2 of 4 graded examples" (1550) | Row 1's own next steps were the note-about-prospect check, the fact-check tuning, then "frozen and run 3 times in the same hour"; the review set is that run (its table shows the note check firing, 0 / 2 / 0), so it is the later result and the ledger row is missing its closing line. | HARD, contradicted on sequence | Date both and say which is later. Report row 1's own exit test: did all four graded cards hold in all 3 runs? The review set never says. |
| Floors: coverage 85%, hand-off correctness 90%, shadow agreement 90% (A1, 627; Floors, 209) | Old page 196 "proposed", no derivation; shadow agreement needs a mailbox copy that Situs does not have (old 190). | ESTIMATE | Call them starting values. Note that both live customers (33% and 47%) sit under 85 while A1's recommended option promises "no change to what renters get". |
| "one in three hand-offs that needed a person never reached one" (The idea, 78; audit, 280) | 121 of 326 (37%) at Camellia, 51 of 172 (30%) at Western Slope, 172 of 498 (34.5%) combined; "Failed" includes failed to transfer or answer; Situs has 2 graded conversations. | HARD, with the figure's own caveats | Advice (b) cites "the Situs and Western Slope rows"; there is no Situs row with a needed-a-person count. Drop Situs from that sentence. |
| Day 0: "there is no frozen baseline yet ... Fede asked yes or no; the answer was no" (579) | Benchmark Day 1: "the frozen episode sets (72 prospect cards, the 20 full-mode repair texts plus the Spanish slice, the 15 graded pairs) ... depend on nothing open" (467); call 14 "(b) accept the 67-card run as the baseline" (1486). | ESTIMATE, partly CONTRADICTED | Say what exists (fixed exports with production numbers) and what v1 adds (hash lock, tiers, the baseline row). State that the repair surface is 20 texts because 215 predate the mirror. |
| Full-context hypothesis "no proof yet" (402 to 404) against "Calls for Fede: (1) strategy B with Opus 5.5 full context (Benchmark D)" (478) | Benchmark 892: same decision on 91% of texts at 2.2 times the tokens, thread-only misread the sender 3 times; Benchmark 98: full mode quoted a two-week-old balance nobody asked for. | ESTIMATE; the recommendation runs ahead of the evidence | Make call (1) "bench Thursday, decide after", or print the 91% and the balance miss beside it. |
| Time per turn: 75 s today, 7.4 s new (Measure table, 394; internals diagram, 376) | Benchmark 510 and 657 (same figures); Benchmark B 17.6 s writer p50 and 29.7 s per turn (744 to 745); Benchmark A 21.8 s (728); the review set 40 s today and 25 s new (716). | HARD (traced) but the sources disagree; the page shows the worst and the best | One measure, defined (writer call or whole turn), with its window. |
| Cost per card $0.29; writer bill about $298 for 30 days (396 to 398) | Benchmark 658 to 659 verbatim, no derivation; Benchmark B $0.48 per turn (744), Benchmark A $0.306 (728). | UNVERIFIABLE (the bill), ESTIMATE (per card) | Give the volume and price behind the bill, or drop the row. |
| Prompt tokens today: 109,658 (388), "~104 to 110k" (376), "about 104k" (131), 103.8k (Benchmark 562) | One Oct 22 trace, four numbers. | HARD, inconsistent | One number. |
| New prompt: about 7,440 median (388), "about 8.5k" (131), "~7 to 19k" (380) | Benchmark 612: "8.5k, prototype B measured ~5k + tools". | ESTIMATE, inconsistent | One number with its basis. |
| "Held, 67 recent cards: 6 today, 0 new" (390) against the review set "Held by a send check: 10 / 6 / 10 today, 4 / 5 / 6 new" (714) | Benchmark 655 for the first; the three-run table for the second. | CONTRADICTED on the page | Use the three-run numbers; the 0 is from a smaller, earlier run. |
| "No rewrite step caught a real problem in 14 days" (gates, 353) against the same table's "check with the team 31 (1 real)" (358) and "Review-mode hold 102 fired, 1 real catch" (362) | Benchmark 1360 to 1385. | CONTRADICTED in one table | "Two real catches in 14 days", or define what counts as real. |
| "9 live checks" (Figure 7, 412; Four interfaces, 421) against "Is Clara live: 8+ sites, three functions, eight named callers, plus raw flag reads" (241) | Old page 519 against old 436 to 448. | HARD, inconsistent on the page | One count, and say whether raw flag reads are in it. |
| "13 live agents" (341, 1826) against "12 ElevenLabs voice agents" (diagram, 376) and "97 app paths, 15 voice agents" (Benchmark 298) | The 0d voice table names 13 live agents. | HARD (13); 12 and 15 are stale | Fix the diagram and the "full table" line. |
| "about 13 paging paths" (412, 425), which the page calls "an Aug 24 estimate the code comment still carries" (245), against Figure 13's 17 ways | Old page. | ESTIMATE, flagged stale by the page itself | Use 17 or recount. |
| 475 tenant messages: "274 answered by nobody; staff answered 189; 1 answered by Clara" (480 to 484) | Benchmark 849: staff 189, nobody 274, AppFolio's own AI 11, Clara 1; the three tiles add to 464. | HARD, with the 11 dropped | Add the 11 ("AppFolio's own assistant answered 11") so the tiles add up. |
| Triage "1,994 real texts: 45% need no reply" (501) against "Tool coverage, 2,591 tenant texts: 34% need no reply" (470) | Benchmark 1005 to 1024 (1,519 Situs history plus 475 recent) and 1413 (2,591). | HARD, two different sets, never said | Say they are different sets and windows, or keep one. |
| "65 staff asks by text have gone to Kat and Ryan since Sep 24" (165) while A2 asks whether Clara may text staff a question at all | Old page 152; old 489 says a text goes to the manager only if teamNotifyText is on (it is off). | UNVERIFIABLE whether the 65 are questions or tour notices | A2 must say which. If they are questions, A2 is already answered in practice and the ask is only about Situs. |
| "Hold on maintenance texts 120 held, 101 expired" (14 days, 364) and "95 of 142 held drafts expired" (Aug 1 to Oct 20, 150 and 299) | Benchmark 1378; old page 225. | HARD, two windows never labeled side by side | Label the window on each number. |
| "Of 27 real first emails, 17 were changed ... 7 worse" (353) against "Emails rewritten after writing 51 of 78" (392) | Benchmark 653. | HARD, two windows | Label. |
| Two sets of six decisions, both numbered 1 to 6: "Decisions for you" (prompt shape, customer rules, model, bar, retry, small-model checks; open) and the ledger's "You answered all six tonight" (hosting, own token, retention, the opener rule, the four fixes, the work-order number; answered) (Decisions for you; Ledgers, 1794) | Benchmark 476 to 485 and 188 to 195; the page's sentence "The Benchmark lane's decisions keep their own numbers (1 to 6)" points at the open set. | CONTRADICTED by numbering | Renumber or retitle one set; a reader can believe the open six are done. The morning list's "switch or no switch" and "model-first or path-by-path" stay open beside them and belong on one dated list. |
| Morning list "five things, in order" (1778) has six bullets; decision 1 "3 datasets, 9 runs" against "four datasets, ten runs" (1774) | Benchmark 180, 190, 178 (carried as is). | HARD, inconsistent, inherited from the source | Fix in both places. |
| G2: "filed 8 of 42 where the office filed, all right" (499) | Benchmark 985 to 989: 8 of 42 (was 11), invented a fact in 1 of 261 (was 13); "all right" is not in the source, and the bench correction says G2 did not run the production prompt. | HARD numbers; "all right" UNVERIFIABLE | Drop "all right" or source it; mark the row as a rules-prompt bench. |
| Benchmark decision 3 "(b) Opus 5.5" for the first-reply writer, carried verbatim (Decisions for you) | Ledgers, filing table, Oct 25 evening: the decomposed classifier on Opus "is not yet at parity"; the safer step is classifier on Sonnet 4.6, writer on Opus. | HARD, consistent only if decision 3 means the writer alone | Say "writer only; the repair classifier stays on Sonnet 4.6 until its thresholds are re-set". |
| "Ledgers from the Benchmark lane", presented as verbatim (1467) | One row added that is not in the source (Oct 25, Slack rating buttons, 1572); done-means row 3 split in two; bullets reformatted; otherwise line for line. | HARD, one addition | Mark the Oct 25 row as added after the merge. |
| "Condensed Oct 25 (parts 1 to 7 cut to under 8,000 words)" (1455) | The stripped page is 34,207 words. | HARD for parts 1 to 7 only | Say the whole page is about 34,000 words, most of it ledgers and the 15 pairs. |
| "The four pages carried 24 open questions" (1338) | Old page 388, same sentence, no list. | UNVERIFIABLE | History; keep or drop, no decision rides on it. |
Traced with no issue: Western Slope has fewer tenant messages than Situs (223 against 252); Figure 5's 2, 2, 8, 2 split of the 14 cases; the 122 silent drops (85 plus 37); Situs at 33% and Western Slope at 47%; the 261-episode table; and the 15 pairs and the four repair cases are line for line with the Benchmark page.
Consistency of the three owner calls: A1 to A3 and their recommendations appear once each and are referenced consistently in "Not scheduled, waits on a decision" and in the appendix. The problem is the count of everything else, not the wording of these three.
Could a builder start on Sunday morning with no questions? Day by day, with the missing inputs.
| Day | Ready to start? | Missing inputs |
|---|---|---|
| Day 0, frozen baseline v1 (Benchmark lane) | Partly | Which exports become v1 (the 72 prospect cards, the 66 pairs, the 20 full-mode repair texts, the 712 inbox messages, the 1,994 triage texts?), the smoke and full tier sizes, where the content hash lives and who signs the baseline row (Fede's bar: today's writer is the floor). The repair surface is 20 texts because 215 predate the mirror; "tenant other text" has no labeled set beyond D3's 23 in-sample examples. Only the first-email surface can start at once. |
| Sun Oct 26, Situs Group onboarding package build starts | No | "Package" is defined nowhere on the page; the hello per role, the ownership map and the week-one tables were condensed away. Situs has no owner on any of its 52 properties and no team email (old page), the ingest pre-fill never saved (66 of 100 policies), there is no kickoff date or named contact, and the channel for staff asks waits on A2. A builder would have to invent the deliverable. |
| Mon Oct 27, model plumbing; Situs package proven at the Willows | Partly | "One model setting per path": which paths first (0d gives the order; cite it on the row), whether the setting is per path in code or one per group, the byte-identical replay set (Day 0), and the voice-line timing test the 0d text requires. Fede's "model-first or path-by-path" ask is open. The Willows proof depends on Sunday's undefined package. |
| Tue Oct 28, core: shouldSpeak PR 2, askTheTeam and askState pass-through, replay | Yes, with one restore | The sites are listed (5 hold sites, 9 live checks, the opt-out withhold, the human-replied skip) and the dependency is merged. Restore the issue numbers the condensation dropped (shouldSpeak PR 2 is issue #22333, depends on #22334) and state that askState's sentence block is the later behavior PR, not the pass-through. |
| Wed Oct 29, hand-off fixes to live behavior on the shared interfaces, Willows run | No | A builder would ship step 2 as written, including the tracked-ask exception; it needs the rewrite above first. "Routing by the ownership map" needs a map Situs does not have; "delivery confirmed" needs a definition for email (no read receipts). Seven live-behavior fixes in one day is at least five PRs under the one-fix-one-PR rule; name the order. |
| Thu Oct 30, full-context vs restrictive bench row; repair path on the decomposed prompt, writer on Opus 5.5, Willows end to end | Partly | Define the two arms (what "restrictive" and "full" each include: lease, unit, work orders, thread, knowledge, settings) and the judge rubric for invented facts. State that the classifier stays on Sonnet 4.6 (its thresholds are tuned for it) and only the writer moves. The 3b writer's team alert (case 4; the all-Opus run) needs an emergency gate before this ships. The Willows run needs a tenant with open orders (the bench tenant has them). |
| Fri Oct 31, tenant replies path, dark: maintenance, then renewals; expired holds become an open approval | No | Depends on calls (1) to (4) (strategy, door, company, who ships in Gera's maintenance lane), on the supervised contradiction, and on renewals' D11 to D17. The hold design is unresolved (card, approval or none). Maintenance and renewals on one day is not a day's work; renewals should leave the week. |
| Sat Nov 1, PmsInbox adapter pass-through; diagram checked against the code; week review | Yes, as a pass-through | The interface's three methods are named (read texts, read staff replies, post a text). State the scope: the inbox mirror reads only; the wider AppFolio literals (about 37 in leasing, most robot files) are later PR chains. The week review needs the Day 0 baseline to compare against. |
Across the week: the release plan's stages 0 and 1 (shadow at three companies, 50 pairs) need no decision yet appear on no day; no day has an owner except Day 0; Fede's own inputs (15 marks, 4 verdicts, A1 to A3) have no date; the English "I'll check with the team" line inside Spanish emails is a named production bug on no day; and the shadow-feed overlay, called "the first draft of shouldSpeak", is not referenced by Tuesday's row.
Independent final pass, Opus 5.5 at max effort, Oct 25 evening, over the 14:14 version of the page (with the Cerebrus section). Findings are the reviewer's; nothing on the page was changed in response yet, except that the "approval queue" finding below is withdrawn. Withdrawn (Fede, Oct 25): "I never said I didn't want an approval queue." His Sept 8 autonomous rule covers identity and household decisions, not a person reviewing Clara's dark tenant replies. The approval card stays as designed.
Not ready to decide from yet. The page puts "add no noise" first, but nothing on it measures noise, and its own plan still lets Clara say "it's with the team", emails staff a notice per question, keeps a daily re-ask with no setting, and puts every tenant reply behind a Slack approval card. The week builds shared plumbing but drops the two Benchmark days that made Clara quieter: the reply-or-silent table, and the fixes for live defects such as the English "I'll check with the team" line inside Spanish emails. Change first: define a noise number (replies a person would not have sent, plus staff messages not tied to an open question), record it in the Day 0 baseline and make it a fourth readiness floor; second, write one rule per channel for when Clara stays silent, and hold the approval card and the per-question notice until that rule exists. Then cut the roughly twenty open decisions to the few that block this week, and fix the three places that still say Fede has three questions.
Ranked by severity. Section names are the page's own headings.
| Item | Quieter or noisier | Why (one line) | Change proposed |
|---|---|---|---|
| No noise number anywhere ("Cerebrus"; "Decisions for Fede", A1) | Noisier: unmeasured | Cerebrus calls the noise rate "the headline number of this page", yet no section defines or measures it, and the three floors count coverage, hand-off correctness and shadow agreement only. | Define it (replies a person would not have sent, plus staff messages not tied to an open question, per 100 inbound), record it in Day 0 and add it to A1 as a floor; the bench's "check with the team" rate (34% today, 8% on the decomposed prompt) is half of it already. |
| The "I've told the team" reply is still allowed ("Fixes ship with no switch", step 2; "Four interfaces", askState; "The internals, re-architected to scale", hard rules keep "the one handoff sentence") | Noisier | Step 2 bans "with the team" only when no ask exists, so the Western Slope reply plus the staff email stays legal whenever an ask is opened, and no section says which channel each rule governs. | One line per channel: a tenant on any channel gets silence and tracking, no sentence (Oct 23, Oct 25); prospect email keeps the one hand-off sentence only when the record shows the ask went out, until Fede rules on extending silence there. |
| Clara talks over staff on her own text line (Benchmark gates, "Also found"; not on the merged page) | Noisier, live as of the Benchmark's Oct 24 read | "On Clara's own text line a staff reply does not stop her from replying" and "84 of 87 AppFolio door turns ended in a hand-off to staff" were dropped in the merge. | Put both back on the page and on Wednesday's fix list; the first is question 3 of "Clara is a hire" failing in production. |
| Every tenant reply waits for a Slack approval ("Tenant communications", proposed path) | Noisier: a queue [withdrawn, see the note above this review] | "Every reply stops at the hold card until a person approves" on every door, including Clara's own line, against "never a queue" and "the customer's own Send button is the only hold left" ("A senior next to her") and round 2's "Clara's own channels answer live or ask with no hold". | Use the page's own supervised rule: in the AppFolio inbox, where staff already do the job, Clara drafts silently and each difference from what staff sent is graded; no card, no approval. |
| A notice per question, and the dark notice engine switched on everywhere ("Four interfaces", askTheTeam; "Clara is a hire") | Noisier | askTheTeam "sends one notice through the team-notify engine (then delete the engine switch)", which turns a default-off staff-email engine on at every customer, while Oct 23 ruled "no escalation out of the inbox: staff already see it". | No notice for anything staff already see in their inbox; other open questions go on the morning list; only the urgency rule sends at once; the engine keeps its default-off setting. |
| Daily re-ask on every open question, no setting ("Where it stands today"; "Fixes ship with no switch", step 3) | Noisier | The live ask record re-asks its owner every day, the page lists it only as something to reuse, step 3 even proves "the chaser" at the Willows, and "Connect, read, learn who's who" promises "she never nags". | Fold the re-ask into the morning list (one list per person per day, empty days send nothing) and delete the separate re-ask. |
| A1 keeps today's replies, and "Pull back" adds put-offs ("Decisions for Fede", A1) | Noisier | The recommended option means "no change to what renters get", so the "let me let the team know" reply continues until a module clears the floors, and "Pull back" promises "more 'I've asked the team'". | State that the silence rule is a fix to live behavior that ships now whatever A1 says; reword Pull back to "topics under the floor stay silent and tracked". |
| Opus on today's prompt says "check with the team" more ("Repair texts", G3; "Upgrade path to Opus 5.5") | Noisier | On 261 real repair texts the share rises from 34% today to 39% on Opus with the same prompt, and falls to 8% only with Fede's fixes. | No writer moves to Opus on today's prompt; the upgrade ships with the decomposed prompt, and Monday's plumbing stays a no-behavior change. |
| Repair writer alerts the team and duplicates orders ("Clara Benchmark, night of Oct 24"; "The goal, and what done means", row 2) | Noisier | On Opus the writer said "ya alerté al equipo" about a non-emergency, handed clear repairs to the team (team items 50% to 64%), and duplicates of open work orders rose from 5% to 11%. | Thursday's repair path ships only with team alerts limited to the emergency list and duplicates at or below today's, checked on the frozen set, with filing thresholds re-set for Opus. |
| New first email puts answers off more ("Your review set"; "Release plan") | Noisier | "Put an answer off" runs 6/5/6 for the new writer vs 3/3/4 today, and pair 12 says "I'll check with the team on which studios are open" when the facts show one open; the stage 1 exit checks facts, holds, tone and Fede's grades, not put-offs. | Add "no more put-offs than today" to the stage 1 exit before any company flips. |
| Staff messages that are not an open question ("Every ask since Oct 22"; "Engineering calls, closed on recommendation", rows 13 and 17; "Fixes ship with no switch", step 4) | Noisier | A reply with a dash "goes back once, then the team is asked"; follow-ups flag-and-post fair-housing hits, and the fair-housing screen's holds were right 0 of 16 times; about 20 notice templates are kept uncounted; meeting notes "become asks to confirm"; Clara proposes PMS settings; expired held texts and missing work-order numbers raise alerts with no recipient named. | Never ask the team about punctuation (track it internally); count each notice kind against the noise number before keeping it; meeting notes and PMS advice go on the morning list, dark. |
| Nothing says which asks may interrupt ("Some things are never hers"; A2) | Noisier by omission | The merge cut the urgency rule to one word, and the old page's evidence stands: "Rossi's 11 emails in 67 minutes went out as 11 notes". | Restore the old rule beside A2 (which only picks the channel): an emergency, a tour today or money due today goes now; everything else waits for the list; a burst from one sender is one ask. |
| Quiet rules the merge dropped ("Changes in this rewrite") | Quieter rules lost | The old page capped each owner at one notice per matter and one list per day with zero chasers on renter repeats, and logged a per-owner counter where a breach is a bug; none is on the merged page. | Restore them as the acceptance test for the ask record; the counter is the staff half of the noise number. |
| Hold queues kept elsewhere ("Every gate between Clara and a send"; "Engineering calls", row 14; "The seven days", Friday) | Noisier: queues | The gates table keeps the review-mode hold (102 fired, 1 real catch) and wants to "fix who works" the maintenance hold (101 of 120 expired), Figure 13 deletes supervised-week holds, and Friday makes held drafts "an approval that stays open". | One answer: no hold on Clara's own channels, shadow and grade where staff work, the customer's Send button as the only gate; if review-mode and supervised-week holds differ, say how. |
| Decision 5, option (b) ("Decisions for Fede", Benchmark decisions) | Noisier | "Short code-built reply + team item" is the reply-plus-team-email pattern the page's first section calls worse than silence, and a reviewer prefers it. | Strike (b) as against the page's first rule; keep (a), with a held first email shown on the morning list, not a separate alert. |
| Texts that carry nothing new ("Your morning list", David; "Full mode, Oct 25 afternoon") | Noisier | "I sent you an email about your inquiry. If texting is easier, you can reach me right here" arrived five hours after the email with no answer in it, and the decomposed writer quoted a tenant's two-week-old balance she had not asked about. | A text goes only when it carries the answer or the tour times; no unasked account facts. |
| Cerebrus option D ("Cerebrus") | Noisier for our raters | It sends 362 flags of unmeasured precision to the shadow-card raters, 87% of what it grades is voice (out of scope), and the section says no option moves the noise rate. | Keep only the "let me check, then nothing" detector (75 flags, already app code, aimed at the anti-pattern), kill the rest, and rate a small sample. |
| The four questions before every reply ("Clara is a hire"); askState; the tenant path's checks block "check w/ team" | Quieter | Quiet on thanks, unknown senders and threads a person already has, and never claims a hand-off that did not happen. | Keep; make question 2 say "track it" with no notice for anything staff already see. |
| Morning list instead of an email per item ("Clara is a hire"; step 4; A3) | Quieter | One list, dark, a company setting Fede turns on, matching "a daily digest is a setting". | Keep; add a per-person opt-out and say it replaces the daily re-ask and Camellia's hand-written note. |
| Teammate check before every send; free code gate on notices, staff-typed and unknown senders; rewrite steps removed ("Engineering calls", row 15; "Tenant communications"; "Every gate") | Quieter | With the teammate rule in code, replies on staff-run threads fell from 10 to 13 per arm to 0 (Benchmark D2), and silent means "nothing sent to anyone". | Keep; these are the first rows of the silence rules. |
HARD: traced to a source number. ESTIMATE: a stated estimate or proposal. INVENTED: stated as fact with no support. UNVERIFIABLE: in neither source extract; it may come from tonight's session and needs its evidence on the page. The Cerebrus section was added after the merge and is checked against itself.
| Claim (section) | Label | Page says | Source says, or why it fails |
|---|---|---|---|
| Hand-off audit headline ("One in three hand-offs that needed a person never reached one"; "Advice on the order") | HARD counts, overstated words | "one in three hand-offs that needed a person never reached one" | Old page: "121 of 326, 37% (36 + 85)" and "51 (14 + 37)", 172 of 498 combined, and "silent drops may be overstated". 122 of the 172 are silent drops where staff may have answered in AppFolio or Outlook, and failed phone transfers are inside the count; as the hand-off floor it rewards asking over silence. |
| Readiness floors ("Decisions for Fede", A1) | ESTIMATE | "coverage 85%, hand-off correctness 90%, shadow agreement 90%" | The old page proposes the same values with no data behind them; the onboarding bar is "80 to 90%". The page says hand-off correctness was "computed once by hand" and mailbox shadow-and-diff "does not exist yet", while Situs sits at 33% coverage and Western Slope at 47%. |
| Day 0 frozen baseline ("The seven days") | Re-scoped; parts UNVERIFIABLE | "a fixed corpus per surface (prospect first email, prospect reply, tenant repair text, tenant other text, Spanish slice across all), smoke and full tiers, content-hash locked"; "renewals and voice get their first frozen set later in the week" | Benchmark day 1: "the frozen episode sets (72 prospect cards, the 20 full-mode repair texts plus the Spanish slice, the 15 graded pairs), the three arms as one command", "judge with controls". Two new surfaces, the tiers and the hashing have no size or source, the three arms and judge are not named, no day holds the renewals and voice sets, and phone is out of scope. |
| Day 0 status ("The seven days") | UNVERIFIABLE | "Fede asked yes or no; the answer was no" | In neither source. It reads as if Fede declined, while the next sentence says he wants the baseline first; quote his question. |
| Situs intro proof ("Advice on the order") | UNVERIFIABLE | "the intro texts, proven at the Willows on Oct 25 from the building line" | Neither source has it; the old page listed the Willows blocker (Fede's number is a teammate in the phone registry, its Willows listing removed Sep 28). Add the message ids or drop "proven". |
| Fede's order overrides the plan ("Order of work"; "Advice on the order") | UNVERIFIABLE | "His order holds and overrides the earlier 'one module a day' cut" | The three-priorities quote is in neither source, and the cut it overrides was Fede's own Oct 25 words ("building module by module, going deep with a session on each", Benchmark 0e). Date and quote the newer instruction. |
| Monday is the upgrade ("Advice on the order") | INVENTED; contradicts source | "the Opus 5.5 upgrade is the Monday model plumbing" | Benchmark day 2: "one model setting per path", "replacing inline model literals, no behavior change". No model moves on Monday; the upgrade comes later, path by path. |
| Full context as the design basis ("Full context, trusted"; "Tenant communications", call 1; "One core engine", Figure 7) | UNVERIFIABLE until Thursday's bench row | "a hypothesis, no proof yet", yet call (1) asks Fede to approve "strategy B with Opus 5.5 full context (Benchmark D)" and Figure 7 draws "full context (hypothesis)" inside the core | Benchmark section 8: "none of the tenant or repair benchmarks below (D, D2, D3, G, G2) ran the production prompt". Move call (1) after Thursday and label the core box "context assembly". |
| Switch on the new writer ("Release plan") | HARD (page's own words); contradicts source | "A new writer is a new capability on a live path, so it ships behind a company setting" | Done-means row 1, verbatim on the same page: "the production PR (decomposed first email on Opus 5.5, no switch)"; the Benchmark page listed "switch or no switch for the new writer" as open. The page decides it quietly against the standing no-switch rule for improvements to live behavior; ask it or follow the rule. |
| "Decisions for Fede holds three questions" ("Where the reviewers disagreed"; "Engineering calls": "three owner calls survive"; "Decisions for Fede") | HARD count; contradicts the page | "Three questions from the lifecycle side come first (A1 to A3)" | Count: A1 to A3, the Cerebrus decision, Benchmark 1 to 6, two release-plan questions, four tenant calls, four repair cases and the David text call, about 21, plus 15 grades. "Decision 6" also means two things: small checks to retire, and the work-order number text in the bench-night table. |
| Prompt-cut proof ("Cheap vs big") | UNVERIFIABLE as worded | "~62k tokens per first reply; identical replay proves it" | Same words in the Benchmark source, which also says "Sonnet matches itself only 3 of 12". It holds only if the cut text never reaches a prompt; if it does, outputs change and no replay can be identical. |
| Core diagram caption ("One core engine"; "Four interfaces") | ESTIMATE, minor mismatch | "5 supervised-hold sites, 9 live checks", "about 13 paging paths"; the interface table drops "about" | Old page: "~13 (Aug 24 estimate quoted in the code, not a live count)"; "Where it stands today" says "eight named 'is Clara live' call sites plus raw flag reads". List the 13 before Tuesday. |
| Cerebrus inventory ("Cerebrus") | UNVERIFIABLE by this review | 3,509 turns graded, 362 flagged red, 0 human labels, 0 CI catches | Neither source has them, but each cites its own query, so not invented. Internal check: the per-detector flags (318 + 13 + 75 = 406) exceed the 362 total; say whether 362 counts turns or flags. |
| Kat and Ryan in the Situs paragraph ("Connect, read, learn who's who") | HARD, wrong context | "65 staff asks by text have gone to Kat and Ryan since Sep 24", given as Situs evidence | Old page ownership map: Kat is Western Slope's leasing owner and Ryan F. her backup. Label it Western Slope. |
| Benchmark ledgers ("Ledgers from the Benchmark lane") | HARD (checked line by line) | Carried from the Benchmark page | Verbatim except one newer asks-ledger row (Slack rating buttons, Oct 25) and connector sentences. They disagree with the body on the switch (done-means row 1), on team items ("No team items in phase 1; silent or helpful"), on the dash rule's team ask, and on the two Benchmark days the week dropped. |
| Prompt size ("One thing: the lifecycle is the product"; "The internals, re-architected to scale") | ESTIMATE, consistent | "about 104k tokens today, about 8.5k proposed"; elsewhere 109,658 vs a ~7,440 median | Benchmark: "8.5k", with "prototype B measured ~5k + tools". Same range on different bases; pick one pair. |
Section: "The seven days, Oct 26 to Nov 1". Ready means a builder can start Sunday morning without asking anyone.
| Day | Ready? | Missing inputs |
|---|---|---|
| Day 0, tonight (Benchmark lane) | Partly; already started | Size and source of the "prospect reply" and "tenant other text" sets; what the smoke and full tiers hold; what locks the hash; whether the three arms and the judge with planted controls are in; the noise count to record. Only 20 of 261 repair texts run with real lookups (215 predate the message mirror), and 8 of 11 turns read today's records, not the records at the time. |
| Sun Oct 26: Situs onboarding package | No | What the package contains (the merge dropped the hello-per-role figure and the ownership map); who receives it at Situs, which has no owner on any of its 52 properties; staff cells and consent; the default-off setting it ships behind, since intros are new outbound in rollout step 4. |
| Mon Oct 27: model plumbing; Situs package at the Willows | Plumbing yes; Willows proof no | Whether the 15 voice agents are in (phone is out); a Willows phone set up as a tester, not a teammate; which message ids count as proof. |
| Tue Oct 28: shouldSpeak PR 2; askTheTeam and askState pass-through | Partly | The tracking issue for PR 2 (the old page had it); the list of the about 13 paging paths; which rules shouldSpeak applies, since the reply-or-silent table is not scheduled. Two interfaces in one day is likely two days. |
| Wed Oct 29: hand-off fixes plus a Willows run | Partly | The fix list is rollout step 2, but the per-channel silence rule and the staff-reply defect on Clara's own line are not on it; the Willows run needs the tester phone. |
| Thu Oct 30: full context vs restrictive; repair path on Opus | No | A definition of the "restrictive" arm; Fede's calls on the four repair cases; filing thresholds re-set for Opus (8 to 43 stale open orders per Situs tenant); a rule that limits team alerts to emergencies. |
| Fri Oct 31: tenant replies, dark | No | Whether replies go through a Slack approval card ("Tenant communications") or no hold (round 2, Figure 13); which setting it ships behind; whether "an approval that stays open" reaches beyond renewals; the renewals packet fill still waits on D11 to D17. |
| Sat Nov 1: PmsInbox pass-through; diagram check; week review | Pass-through yes; review no | Which numbers the review reports; propose the noise number, the "check with the team" rate and staff messages per open question. |
| Missing from the week | Not scheduled | The reply-or-silent table (Benchmark day 6); the live-defect fixes (Benchmark day 4); Clara replying over staff on her own text line; what happens to the daily re-ask. |
Fede (Oct 25, voice): "should we kill it or should we merge it?" This section is the inventory, read from main on Oct 25, and the decision. Spelling: Fede ruled "Cerebrus" (the repo's spelling) on Aug 27; his voice note said Cerberus. Inventory by four read-only workers over the worktree, GitHub, Sentry and the production table; every number names its source.
propflow-prod, source turn_checks_posthoc.MachineGrade row per turn (turn_checks_posthoc, engine v0.1.19)turn-complete-dispatch.ts, conversation-grade.ts, turn-score.ts, grading/cerebrus/engine.ts. Zero files on the send path import the engine. The in-app thumb writes a different record (GradeVerdict) in the same table; the two share only the conversation id.| Piece | Where | Since | State on Oct 25 |
|---|---|---|---|
Engine repo PropFlow-Technologies/cerebrus | 35 PRs, all Aug 21 to Aug 30 | Aug 21 (#6039) | frozen last commit Aug 30; no PR since |
Vendored copy quality-gate-poc/, pin v0.1.19, checksum, sync script | app | Aug 20 (#6009), re-vendored Sep 2 (#6803) | unchanged 31 fixtures, newest Aug 29; vendor-sync check not run in CI |
| Required CI check "Regression Gate — Leasing (required)" | branch protection on main | Aug 21 (#6045) | blind by construction it re-runs the engine over 31 recorded drafts; a prompt or tool change cannot alter its input (engine imports nothing from agents/clara). 0 failures ever outside planted tests (Aug 29 inspection: 1,286 runs, 2 fails, both deliberate) |
| Advisory gate and judge lane | CI | Aug 21 | deleted Oct 2 (#15238) |
| Parity step (vendored detectors ≡ app checks) and two drift tests | Lint job, continue-on-error | Sep 7 (#7317) | green 0 of 31 sampled runs printed a failure |
| Bridge + per-turn checks in the grading worker | grading/cerebrus/engine.ts, turn-checks/, ECS renewal-worker (2 tasks) | Sep 7 (#7317); first row Sep 8 | live daily newest row Oct 25 08:48Z |
| D8: thumbs-down → fixture → cerebrus PR → Smith hand-off | d8-pipeline/ | Aug 23 (#6123) | dead nightly sweep deleted Oct 2 (#15253); 7 fixtures landed in total, 0 merged tests in the final Aug 30 sweep |
| Quality Desk | app | Aug | retired Aug 29 (#6539); "the conversation is the desk" |
| Sentry | 90 days | 0 issues naming Cerebrus, turn-checks or the grader |
gh on both repos, branch protection API, CI run sample (32 runs Sep 1 to Oct 25), git log, Sentry search, Aug 26 to 30 session notes. Transcripts older than late August are in S3 and were not restored.Every detector judges what Clara said. None judges whether she should have spoken. Cerebrus cannot move the noise rate, the headline number of this page, under any option; that is the honest answer to the lens Fede asked for.
| Cerebrus concept | Covered elsewhere today? | Verdict |
|---|---|---|
| Claim grounding (figure, time, "I already did X" with no tool behind it) | Send path: hallucination guard blocks it before send. Bench: invented-policy grader. Cerebrus: post-hoc, 289 of the 362 red rows carry one (311 findings: 161 times, 145 figures, 3 mismatches, 2 action claims) | duplicated three separate implementations, no shared code |
| Narration leak ("our system", backend talk) | Send path: narration strip and internal-prompt strip. Cerebrus: 13 flags post-hoc | duplicated |
| Unbacked promise | Send path: unbacked-promise check and promise-backing policy, both block. Cerebrus has the detector but the bridge never exports it | dead in Cerebrus |
| Intent left unfulfilled ("let me check" then nothing) | Only here, and written in the app, not the engine: 75 flags | unique keep |
| Planted-defect regression gate | Bench controls: 47 planted mistakes, 47 caught (Benchmark A). Cerebrus's gate: 0 real catches | dead the bench does it |
| Vendored provenance, checksum, pin, attestation | Nothing else needs it; the burn-in's 6 bypasses were never closed | dead weight |
| Machine half of the grading record | Exists: MachineGrade, written daily. The Bench already draws it as "candidate flags". Nobody has rated one | unique but unproven |
quality-gate-poc/src and turn-checks/.| Keep | Delete | One week costs (estimates) | If wrong, Fede loses | |
|---|---|---|---|---|
| A. Kill | Planted-defect idea stays in the bench, where it already lives | Engine repo, vendored copy, sync, pin, parity, drift tests, required check, the per-turn worker checks | 2 to 3 days: one deletion PR chain, branch-protection edit, removal replay (Constitution IV.5) | The only grader that reads every production turn. Re-adding it later is a rewrite |
| B. Merge into the send path | Grounding, narration and promise detectors as blocking checks before send | Engine repo, CI gate, post-hoc worker | 1 to 2 weeks plus a 30-day replay of every real message it would hold (III.4); reverses Fede's Aug 20 "pre-send gate off" | Holds on good replies from regexes whose precision nobody measured; the send path already blocks the same three things |
| C. Keep as CI gate only | Required check and vendored engine | Post-hoc worker checks | 0 days, then CI minutes on every PR forever | A required check that cannot see a Clara change keeps saying green |
| D. Fold: keep the grader, drop the machinery chosen | The 7 detectors as plain app code in turn-checks/, still writing the daily MachineGrade row; its red flags become cards in the internal Slack channel with the same four rating buttons the shadow cards carry | Engine repo (archive), vendored copy, sync, pin, checksum, parity, drift tests, the required check, D8 remnants | 3 to 4 days: inline the detectors with a byte-identical replay over the 3,509 rows (replay:turn-checks exists), delete the rest, point the bench read at the red rows | 3 to 4 days on a grader whose flags may prove to be noise; if so it dies at the 30-day review below |
MachineGrade row. Fede, in his words: "We are using labels in slack now moving forward." Then, later the same evening, on the backlog: "the Cerebrus flags, we should just throw them away since we're doing a new tool and we have more data now." So the 362 existing red flags are discarded: no review, no cards, no labeling step for them, and the voice-versus-text card question is withdrawn. Open, Fede's call: from today on, when a detector flags a new text or email reply (a read-only replay of the last 14 days says about 8 a day: 110 cards, 76 email and 34 text, flagged within 2 to 3 minutes of the reply), does it post a Slack card with the same four buttons (Should have stayed quiet, Wrong, Fine, Better than staff) so a press lands in GradeVerdict, or do the detectors keep writing their quiet machine row with nobody reviewing it? Recommendation: post the card; a detector nobody reviews is deleted under Constitution VII.8 and VII.9, so without cards the fold is a kill with extra steps. If cards: by Nov 25 the Bench reports agreement between new flags and presses, and detectors with no press by then go.Under every option the noise rate is untouched; the quiet-or-reply decision is shouldSpeak in Figure 8, not Cerebrus. Fede's Aug 20 ruling (no pre-send gate) stands unless he picks B. The branch-protection edit (dropping the required check) is a GitHub settings write done in the build, right before the deletion PR merges.
Build (session 019, started Oct 30): PR 1 #24099 (merged Oct 30 night) moves the detectors into app code; proof: a read-only replay of 3,048 real Clara turns since Sep 30 (504 calls, 1,045 texts, 378 would-flag) gives identical output before and after · PR 2 #24112 (merged Oct 31 13:05 MT: 361 files, 35,456 lines removed, grader output byte-identical on the same 3,048-turn replay; also removes the quality-replay harness, which only compiled against the vendored engine) deletes the vendored copy, sync script, checksum, parity step, drift tests, D8 remnants and the CI jobs; the required check "Regression Gate — Leasing (required)" was removed from branch protection on main Oct 31 07:45 MT (required checks now: Build, Type Check, Unit Tests, Lint, review). Its first CI run died in the review classifier on the size of the file list ("spawnSync gh ENOBUFS"); the one-line buffer fix #24117 merged Oct 31 08:55 MT, its checks were re-run and it is armed to merge on green (dangerous-diff label: the review workflow and merge guard edits are comment and name-list changes, read in full by this session) · PR 3 (branch feat/cerebrus-fold-3-red-flag-slack-cards built and pushed Oct 30 night, unit-covered, 619 lines; held: no card posted, no PR opened until Fede answers the open question above) posts each NEW red text or email flag as a Slack card with the four rating buttons, never the backlog. One more thing for Fede before it goes live: the shadow cards' channel (C0C283G9QDR) is tagged in the repo as Western Slope's activity feed, audience "customer", though today only Fede, Gera, Sean and the Smith bot are in it; the cards would carry Camellia and Situs conversations too, so they should go to a channel that is ours by name (one-line change). Recommendation: a new internal #clara-ratings channel · the cerebrus repo PropFlow-Technologies/cerebrus was archived Oct 31 13:10 MT (read-only, last push Aug 30, no open PRs; its one open issue, #36, describes the "stated intent, no tool call" idea the app already runs as intent_unfulfilled). Proven in production, Oct 31 13:57 MT: the grading worker redeployed from the deletion commit at 13:21 MT (deploy green) and by 13:57 MT had written 14 per-turn rows on a real Western Slope call, rubric turn-checks-v4+cerebrus-v0.1.19, same as the last row before the deploy (12:37 MT). Steps 1 and 2 of the fold are done; step 3 waits on Fede's two answers above. Closed, Nov 2 (Fede, via 007: "I thought we were done with Cerebrus", "I don't want to continue with it"): the held branch for Slack rating cards on new flags was deleted, no PR opened, no card ever posted. The seven checks stay as quiet app code writing their machine row; nothing reads it. No further work on this lane.
| Group | Paths | Real weekly volume | Model today |
|---|---|---|---|
| Clara's main text and email reply | 1 (plus 3 pieces already moving) | ~120 real turns | Sonnet 5 |
| Voice line | 13 live agents | ~61 real calls | Sonnet 4.6 |
| Customer-facing writing | 19 | small, mostly under 20 a week each | Sonnet 4.6 / Haiku 4.5 |
| Decisions and classification | 59 | two biggest ~200 a week each | Haiku 4.5 |
| Internal writing | 15 | small | mixed |
Nothing in production runs on Opus 5.5 today (real traffic, Oct 18 to 25). Fede's question "why do we need topic classification?": we don't, for phase 1; the turn itself decides reply, silence or hand-off. Path by path: model paths.
Of 27 real first emails, 17 were changed after Clara wrote them; 7 changes made the email worse. No rewrite step caught a real problem in 14 days. Fede, Oct 24: "re-inspect all the gates to see which ones are working and not working" and "we are also removing the rewrite gates".
| Gate | 14 days | Action |
|---|---|---|
| Rewrite steps on the first email | "check with the team" 31 (1 real); tour times 5 of 5 wrong; booking promise 12 of 12 wrong; hallucination guard 8 of 10 wrong | Remove |
| Fair-housing screen (Haiku) | 0 of 16 holds right | Retire last |
| Review-mode hold | 102 fired, 1 real catch | Keep |
| Hold on maintenance texts | 120 held, 101 expired unsent | Fix who works it |
| Door wake rules, voice blocks (482), staff notice check | working | Keep |
The Benchmark lane's definition of done: done means.
bash ~/agents/006/proto/run.sh, then open http://localhost:8766/. Read-only data copied from the audit; nothing calls production.publishInboundMessagelambda/inbound-processor → processEnvelope (text, mirror) or processEmailRecord (every email lane)decideSupervisedReplyHoldresolveActor) → tour pipeline → agent turnrouteInboundMessagehandleIncomingMessageTurn: one system prompt (getUnifiedSystemPrompt), tools by capability, model, tool looporigin/main. Green boxes are one piece of code each. Amber boxes are where the same question ("should this go out?") is asked again in a different file. Voice is its own lane: the live call turn runs inside ElevenLabs and only the transcript comes back through the queue.