Decision brief for Wednesday, 9/24. Updated 5:00 AM MT with what shipped overnight. Proposed — pending Fede.
2026-09-23, written for the morning of 2026-09-24. Live numbers counted from production at 8:15 PM. Status: proposed, pending Fede.
Written 2026-09-24, 5:00 AM MT; updated 9:00 AM MT. Production read-only at 4:30 AM MT. Production is running the build from 3:19 AM MT (d89786be1).
Twenty-four changes merged. Grouped by what a person would notice:
| What changed | Why it matters |
|---|---|
| Clara stops saying things the record does not support. She refuses a pet claim that contradicts the property record; she refuses a first email naming a tour time that has already passed or a policy nobody wrote down; she never quotes a renewal rate for paperwork that does not exist; she respects the budget the person put on their own card. | This is the fabrication class — the failures that would embarrass us in front of a renter. |
| A bad draft can no longer slip through quietly. A draft Clara's own read-through flagged as defective is now marked as defective when it reaches the review queue instead of looking like any other draft. Held drafts now show up on the review tab. Approving a draft that has gone stale rewrites it instead of sending a dead tour time. | The reviewer sees what is wrong before pressing send. |
| Every inquiry is treated as its own lead. A second inquiry from the same person on a different home no longer disappears behind the first. Clara is assigned across every open inquiry on a card, not just one. A lead files against the right unit even when the building's street number differs. | Leads were being silently merged and lost. |
| An email that cannot be threaded is held, not sent. Previously a draft with no conversation to attach to fell straight through the review gate and went out unseen. | The worst case: the reviewer believes the hold is on and the prospect gets the mail anyway. |
| Phone-side fixes. Blind-transfer detection, ring-list handling of unclear answering-machine verdicts, the California AI-disclosure line on outbound renewal calls, the home a call actually quoted resolved correctly, renewal retries capped. | Mostly renewals and voice; not on the Western Slope email path. |
Merged: #11208 #11160 #11156 #11473 #11413 #11403 #11342 #11372 #11425 #11288 #11269 #11426 #11410 #11433 #11399 #11393 #11475 #11227 #11277 #11417 #11525 #11412 #11550 #11557. Closed without merging: #11385 (the pet fix was superseded by #11399 and #11525) and #11292 (folded into #11414). Two of the twenty-four are not in production yet — a test script and a deploy-retry fix, neither of which changes anything Clara does.
| Work | State |
|---|---|
| Clara's first email books the tour it promises (#11414) | Open, fourth review round on the same seam (the "approve a stale draft" path). The reviewer asked for a design change rather than another patch, and that is what is being built. If the next round is red again, this one comes to you. |
| A safe way to rehearse an autonomous first email at our test property (#11657) | Open, built overnight, one review round in. Its first run found a real bug (next section). |
| A lead whose contact is already claimed by a different person now pages instead of retrying silently | Being built; ships live as a bug fix. |
| A lead parked in the delivery queue pages the team | Being built; ships live as a bug fix. |
Merged since 5:00 AM: #11468 (the renter's pet answer labelled as theirs) and #11409 (structured listing facts beat marketing text). Both have proof runs in flight at Willows. Twenty-six merged from this lane overnight.
Read straight from production, read-only, nothing written. Western Slope first-email review holds since 9/20, split by whether the draft was written before or after the first of last night's fixes went live (4:30 AM UTC, 10:30 PM MT).
| Drafts written… | Approved as written | Edited first | Rejected | Still waiting | Total |
|---|---|---|---|---|---|
| before last night's fixes | 11 | 0 | 10 | 1 | 11 + 10 + 1 = 22 |
| after last night's fixes | 0 | 0 | 0 | 0 | 0 |
That zero is the whole story. The newest Western Slope draft was written at 7:47 PM MT last night; the first fix went live at 10:30 PM. No prospect has come in since. So none of last night's work has been tested against a real Western Slope renter, and the thirty-draft bar has not moved. The first real inquiry of the working day is what starts the count.
No prospect names here by design. Source: escalation-hold rows for Western Slope, first-email kind, on the entityType-createdAt-index, created on or after 2026-09-20. "Edited first" counts drafts whose text was rewritten before approval; none carry that mark.
The rehearsal tool itself (#11657) is in review. Its first run at 5 AM failed, and the failure was a real bug, not a test problem: the test identity's email was already on file for a leftover person from a 9/19 test, so the lead merged onto the wrong record, failed the identity check, and sat retrying in the delivery queue, where a parked lead pages nobody. Three bug fixes came out of it, all shipping live with no switch:
Could the bug drop a real Western Slope lead today? Not yet: every Western Slope lead since go-live delivered. But about 30% of the imported Western Slope contacts already carry a failed identity match, so a household sharing an email, a couple, or a returning renter would have hit it. Separately, 17 Western Slope leads are parked right now from a 9/23 backfill of year-old inquiries (Nov 2025 to Jul 2026); see the decision below.
One thing the rehearsal left open. The AppFolio "Email Sent" note with the email text has not been read back green yet; the write-back had not landed when the check ran. (The second run's "not mirrored" was the test's own doing: it reused the first run's phone number, so the engine correctly treated it as the same person. The card was mirrored in 46 seconds. For real Western Slope cards since 9/21 the median time from card to mirrored lead is about a minute and a half, the slowest just over six minutes.)
Still true: the Willows Outlook mailbox needs a person to sign in to Microsoft (the rehearsal used our own Microsoft 365 test inbox instead); the browser robot's sign-in to the test database loops (#11541). Review mode on the test company was already off; nothing was flipped at Western Slope.
Two switches, both on the same back-office page, both platform-admin only.
| Switch | Where | Now |
|---|---|---|
| Clara replies to email (the go-live switch) | /admin/companies/<company id>, card titled "Clara replies to email". Stored as claraLive on the company. | Already on for Western Slope. |
| Hold every draft for a human (review mode) | Same page. Stored as reviewOutboundEmails on the company. | On for Western Slope since 9/21, 12:26 PM MT. This is the one that turns autonomy on when it goes off. |
Both are company-wide; there is no per-home version of either. Turning review mode off means every Western Slope first email sends the moment Clara writes it. Turning it back on is the same page and takes a minute. Neither was touched.
Correction to the overnight finding on named homes: the AppFolio card's home IS read and used (142 of 198 Western Slope inquiry records carry the unit id; the rest are cards where AppFolio sent no unit id, all on apartment buildings). The real defect was the listing-facts block being fed two fields the card path never writes, so it showed the whole roster next to the correctly named home. It is fixed in the rebuild, not patched.
Should Clara send her first email to a Western Slope prospect without a person pressing send?
Today every first email Clara writes to a Western Slope prospect waits for a human to press send. That wait is the review hold. The question is whether to remove it so Clara sends on her own.
Three questions, in order.
Decision rule. Embarrassing failures: zero tolerated. If the overnight rerun still shows an embarrassing pattern that is not fixed, keep the hold one more day. Bland misses: ship anyway and fix them in flight. A low as-is score made only of bland misses is not a reason to hold.
The 7 PM rerun finished. Same 205 real Western Slope guest cards, same judge, on tonight's merged code.
| Verdict | 7 PM | 5 PM |
|---|---|---|
| Send as-is | 94 | 93 |
| Needs an edit | 38 | 25 |
| Do not send | 69 | 87 |
| No email written at all | 4 | 0 |
The 4 punts are new and they are the system behaving: an availability read timed out mid-run and Clara refused to write rather than guess. Counted as not sendable.
The headline barely moved. What is left over changed character completely.
| Failures | 7 PM | 5 PM |
|---|---|---|
| Embarrassing — wrong home, wrong price, tour on a gone home, made-up fact, doubled greeting | 5 | 17 |
| Bland — thin or generic, but not wrong | 102 | 95 |
The embarrassing five: tour on a gone home 7 → 4 (all four are the same shape — the property genuinely has nothing open and no alternative to offer), doubled greeting 9 → 1, made-up fact 3 → 0, wrong home and wrong price still 0.
The bland rose because the fixes traded one failure for another. Reciting the listing went 29 → 0. Replies got shorter — 88 words to 66 on average — and a big block slid from "says too much" into "says too little": a bare tour ask that names the home but gives no rent, beds or availability went 9 → 44, and is now the biggest remaining group. That is thinness, not a trust problem.
Code: main 045f27da7b plus #11344 and #11371, integration HEAD 392e3d3b0d. Judge unchanged from the 5 PM run.
This is the new path, not the composer above: every guest card in the shared corpus replayed through the guest-card-as-inbound-message engine (the code merged today as #11827, #11880 and #11904) in the sandbox mirror of Western Slope. Nothing was sent; nothing was written to Western Slope. The judge is the same one as the runs above.
Corpus sizes are what exists, not what was assumed. The shared corpus holds 50 Western Slope cards with a PropFlow identity to compose against, not 205, and only 1 of those 50 has a held draft in the review table, not 23. The 205-card set belongs to the older composer run above.
| Dataset | Rows | Wrote a reply | Chose not to reply | Errors | Send as-is | Needs an edit | Do not send |
|---|---|---|---|---|---|---|---|
| Western Slope guest cards | 50 | 47 | 3 | 0 | 22 (44%) | 1 | 24 |
| Western Slope held first-email drafts | 1 | 1 | 0 | 0 | 1 (100%) | 0 | 0 |
| Camellia guest cards, graded against Western Slope's company answers | 170 | 165 | 5 | 0 | 78 (46%) | 2 | 84 |
Does the engine meet the bar? No. Western Slope cards: the bar was 0 do-not-send and 90% or better send-as-is; the engine scored 24 do-not-send and 44% send-as-is. Held drafts: the bar was 90% approved-as-is with 0 policy or escalation misses; the engine scored 1 of 1 approved with 0 misses, but one row proves nothing. On the four questions asked yesterday the engine was clean: right home named, policies and figures, calendar and escalation each graded 100% on every reply where they could be graded.
Where the 24 Western Slope rejections come from (a row can be in more than one group):
| Group | Rows | What it looks like | Kind |
|---|---|---|---|
| Tour ask on a home Clara says she cannot confirm | 11 | “I want to double-check its current availability with our team… I've got tomorrow at 10:00 AM open.” Rent, beds and pets are answered correctly; availability is hedged and a tour is offered anyway. | Engine defect, confirmed. The sandbox mirror does carry fresh availability for every one of these homes (written today), and for 566 Placer Street it says rented, move-in 10/1, so the home is gone. The engine only checks availability on the path where it has to work out which home the renter means; when the home is already named on the card it skips the check and hedges. A fix PR is being built tonight: one availability decision, used on both paths. |
| Names no home | 9 | The reply answers without ever naming the address the card carries. | Partly the two-home cards that #11933 fixed after this run started; the rest is an engine gap (name the home in the first line). |
| Names a different home | 5 | Three are the grader misreading “Northridge Apartments” versus its street address, 565 Redcliff; the reply is right. Two name a home from a neighbouring card. | 2 real, 3 grader. |
| Home is gone, no alternative offered | 2 | “239 S Elm Street is no longer available. What size home and area are you hoping for?” | Engine gap: offer two open homes instead of a question. |
Camellia and Yale 25 Station as the do-not-break check (Fede: one implementation for centralized and on-site teams; Camellia and Yale 25 cannot break). Camellia's 170 cards were composed against Camellia's own property knowledge in the sandbox (its home, units and answers were already seeded there); only the company-level answers in that sandbox are Western Slope's, so its row is a usable regression row with that caveat. Yale 25 Station has no guest-card inquiries in production at all: the guest-card sync has never run for that property, so it has nothing to replay and no exposure on this path until guest cards are turned on there.
Camellia's 84 rejections are a different story and mostly not about this engine: 33 ask which size the renter wants when the card already says it, 27 open with “Thanks for your interest” filler, 22 name no home because Camellia cards carry no home. Those are graded against Western Slope's company answers, so they say how the engine handles a single-building client, not how it will do at Western Slope.
Known holes on the arm line: the send-time body guard (#11936) does not reject a timing promise such as “will confirm shortly” on an unconfirmed tour, and the listing-recital and listing-fact-contradiction checks do not run on the engine path. In this replay no tour came back as merely proposed (the sandbox has no tour-shower person, so no booking was attempted on any of the 221 rows) and 3 replies carried a timing promise, all 3 already rejected for the availability hedge above.
Big for the morning, in order: (1) the availability hedge above if the fix PR does not land tonight; (2) the escalation-claims policy gate still defaults off on production, and turning it on tonight would hold every confirmed-tour first email, so it needs the planned tour passed as backing first; (3) the listing-recital and listing-fact-contradiction checks are still composer-only and do not run on the engine path.
Fede, verbatim, the whole spec (6:15 PM): “It's really simple. A guest card arrives, or an inquiry, we look at what property they inquired about, that always comes on AppFolio, we always have that property, we have the ID, we check if the property is available, if it's not available, we offer alternatives, if it is, we answer any questions they might have, if they don't have any questions, we offer tour times, that's it, it's not rocket science.” Order matters: property by AppFolio id → available? no → alternatives; yes → answer their questions from the knowledge base; no questions → offer tour times.
Earlier tonight (5:50 PM), same recipe: the card's property is the home; a unit id is a bonus, never a requirement. Then availability from that property's units (one unit = it; all occupied = gone, say so and offer alternatives, no tour of it; any vacant = a confident answer with the price). Then tour slots from the property's calendar. Then the knowledge base for any question the card asks. Nothing else in the compose path; never hedge or forward to the team because a unit is missing. Tonight's fixes are being cut to exactly this recipe, and the arm ask goes to Fede in the morning with the final before → after table.
Fixes cut to that recipe tonight (each a live bug fix with a mutation control): #11968 merged, availability checked for a named home with a unit id; a second PR in flight extends it to every card by property (occupancy by id decides available or gone, the price shown is the listing price, never market rent); #11974 stops the escalation gate from swapping a first email for the “noted for the property team” template and removes the internal wording from that template; #11967 makes the first sentence name the home from the record; #11916 merged, greeting by the renter's name. The three “wrong home” rows turned out to be a stale corpus join, now rebuilt (1,116 Western Slope rows over 908 cards), not engine defects. A read-only check of Western Slope's synced data: all 40 corpus properties carry fresh occupancy by AppFolio id; only 3 carry a move-in date and advertised rent on their vacant units.
The 50 Western Slope cards were replayed again on main plus the three fix PRs (#11968 availability decision, #11967 name the home, #11966 grader alias). This rerun does not count yet: it was cut before the greeting-by-name fix (#11916) merged, so 5 of its rejections are replies that open “Hi there” and ask the renter their name, which main already fixes. The final rerun happens on merged main once #11968 and #11967 land.
Branch run (all four rules + the ≤3-open rule) in progress; number lands here when judged.
| Western Slope cards | First run | Interim rerun 1 | Interim rerun 2 (6:05 PM) | Rerun 4, sandbox at prod shape (8:35 PM) | Rerun 5 — main + #11985 (home facts in the prompt), graded by the fixed judge (#12024) |
|---|---|---|---|---|---|
| Send as-is | 22 of 50 | 21 of 50 | 29 of 48 | 14 of 48 | 27 of 43 |
| Needs an edit | 1 | 4 | 1 | 0 | 3 |
| Do not send | 24 | 24 (5 are the missing greeting fix) | 18 | 34 | 13 |
| Chose not to reply | 3 | 1 | 0 | — | — |
| No reply (run error, not graded) | — | — | — | — | 5 (login token revoked mid-run; retry running) |
| Names the right home | 33 of 47 | 40 of 49 | 47 of 48 | 43 of 48 | 41 of 43 |
| Offers the calendar correctly | — | — | — | — | 43 of 43 |
Different ruler: after rerun 4, #12024 fixed 4 false-positive shapes and added “asks preferences while 1–3 homes are open,” so 14/48 and 27/43 are not the same ruler. The writer baseline is being re-graded with it and replaces 37 of 48 below. Left: asks a move-in date the card has (5), asks size or layout with 1–3 homes open (2), says none is open when one is (2), names no home (2), invented pet fees (1), promises a follow-up (1).
Rerun 2 (48 cards after the corpus was rebuilt; all four fixes, including the greeting one): 13 of the 18 rejections are one shape. Clara says she cannot confirm the home is available, “loops in the team”, gives no price, and then still asks for a tour. An independent inspection found why: the sandbox copies of the properties are missing the field the engine checks before it trusts availability data, and the engine has no fixed “look up the property, is it open, what does it cost” step of its own; it only has that if the model decides to call a tool. Both are being fixed now (harness first, then a shared availability step that the current writer and the engine both use). The other 5: one contradicts the pet policy, one invites a $600-budget renter to tour a $2,495 home, one hands an eviction disclosure to a human twice, one calls a home gone with no alternatives, one volunteers an internet-speed gap.
Rerun 4 (8:35 PM), the first run against a sandbox that matches the real client: after the parity work (#11987) the sandbox copies of Western Slope's properties carry the real leasing settings, tour hours, hand-off destination and company settings, so for the first time the engine saw homes that are genuinely gone and a calendar with real tour times. The number went down, from 29 to 14 of 48. The reason is one shape: 10 of the 34 rejections say the home is gone and then offer no other homes, only a question back (“let me know what you're looking for”). The current engine has no step of its own that looks up the property's open homes and the alternatives; that step is the home-facts wiring in #11985 (and its shared decision module #11984, already merged). Rerun 5 on main plus #11985 is running now to test exactly that. The next shapes: 8 replies recite the listing back to a Zillow or Rent. inquirer who already read it; 5 replies on Northridge Apartments never name the home; one offers a tour of a home with nothing open; 10 are one-offs. Across all 117 inquiries on these 39 cards the engine was 21 send-as-is, 83 do-not-send; right home 100 of 111, policies 107 of 111, calendar 110 of 111, escalation 111 of 111.
Run: 3 shards from a detached checkout of main ca9cbc1361 (includes #11987 parity), sandbox re-provisioned first (readiness: GO, real tour times available), output tmp-artifacts/rerun4-id48-s1..3. The 200-row slice of the full corpus was stopped on this build and moves to the #11985 build.
Correction: the first two columns' “names the right home” were published earlier as 100%; the merge script read the failure list wrong. The corrected figures are above.
Card by card the movement went both ways: 5 rejections became send-as-is, 5 approvals became rejections, 14 stayed rejected. The availability hedge went from 11 rows to 7, so #11968 fixes some but not all of them; 6 rows are a new shape, a reply that says only “I've noted that for the property team” with no price, availability or next step; 3 name a home from a different card. Both shapes are being traced to their mechanism now, and the page will say whether they are engine defects or sandbox artefacts.
Run: 12 shards from a detached checkout of main 29e32c3f22 plus #11915 and #11916, output tmp-artifacts/redrive-engine-full-s1..s12, merged with merge-shards.mjs. 220 composed, 220 judged, 0 errors.
For comparison, the same 48 identity cards were also run through the first-email writer that Western Slope, Camellia and Yale 25 Station use in production today, once on main as it is and once with the availability-range fix (#11976) added. This is the number the engine has to beat, not just the bar.
| Western Slope cards | Today's writer, main as-is | Today's writer + #11976 | Engine, rerun 2 |
|---|---|---|---|
| Send as-is | 37 of 48 | 36 of 48 | 29 of 48 |
| Needs an edit | 6 | 4 | 1 |
| Do not send | 5 | 8 | 18 |
| Names the right home | 46 of 48 | 46 of 48 | 47 of 48 |
| Offers the calendar correctly | 48 of 48 | 48 of 48 | — |
What was corrected: the first grading of these runs called 5 replies “wrong home” because the card says “Pomona Park Townhomes” or “Northridge Apartments” and the reply says the same building's street address. The engine's grading already knew the address; the writer's did not (#11988, harness fix). The four runs were re-graded without re-writing anything. The writer's 5 remaining rejections: a tour pushed on a home three or four times the renter's stated budget (2 cards), two other homes listed beside the card's own, a price stated twice, and a tour offered without naming any home. The range fix (#11976) trades two of those for three new ones, all wording (a unit the renter did not ask about, “I don't have square footage on file”, “I can't speak to internet speeds”); it does not change what the writer gets right about availability.
Run: 2 shards each from a detached checkout of main (writer A) and main plus fede/ws-named-home-fallback (writer B), output tmp-artifacts/composerA2-ws-s1..2 / composerB2-ws-s1..2, re-graded into composerA3-* / composerB3-*. 48 composed, 48 judged, 0 errors in each.
No. Fede asked. A read-only comparison of the replay sandbox against the real Western Slope and Camellia setups found the homes and units match, but most company-level setup does not, and several gaps change what a first email says. Rerun 3 was stopped and restarts once the sandbox is provisioned to the real shape.
| What | Western Slope sandbox | Camellia sandbox | Changes the first email? | Fix |
|---|---|---|---|---|
| Homes and units | The list of homes and units matches. A couple of homes have drifted: one price is off, and one unit's "shown online" flag is wrong. | The list of units matches exactly. | Yes. A wrong price can get quoted. | Refresh the copy of homes and units right before every test run, and add a check that catches drift. |
| Whether the setup can be trusted | The sandbox trusts its data the same way production does. | Production trusts its data one way; the sandbox trusts it a different way, because it is missing a setting production has. | Yes, and it carries into the next two rows. | Copy the property's leasing setup along with the home. |
| The setting that picks where availability comes from | Matches production. | Production points at one source; the sandbox has no setting at all, so it falls back to a different source. Both end up listing the same open units, but production also gives move-in dates and the sandbox does not. | Yes. Move-in dates and whether a tour gets offered can differ. | Same fix as the row above. |
| The knowledge Clara draws on | Company-wide answers match. But several homes lose amenities, deals, tour links and website links in the sandbox, and many other homes get a blank entry where production has none. | Home-level answers match. But the sandbox sits inside Western Slope's setup instead of its own, so it inherits Western Slope's company-wide answers instead of having none. | Yes. A Camellia reply could quote Western Slope's fees and pet rules. | Give this client its own separate sandbox, and copy every home-level knowledge answer. |
| Who shows tours, and when | Production has a named person who shows tours plus real tour hours, tour length and gap between tours. The sandbox has none of that and falls back to a generic stand-in. | Production has its own tour calendar and a shorter tour length, with no same-day weekend tours. The sandbox has none of that. | Yes. The tour times offered can differ from what is real. | Copy the real tour settings, and fail the readiness check if they don't match. |
| Who a hard case gets handed to | Production has a named owner and a list of people to hand off to. The sandbox has neither, so a hand-off finds nobody and Clara says she couldn't confirm anyone was told. | Production is set up to hand off on its own, with a team email and an owner. The sandbox has neither and falls back to the same "couldn't confirm" reply. | Yes, on every reply that needs a hand-off. | Give the sandbox a safe stand-in hand-off address and copy the real hand-off setting; real outreach stays blocked either way. |
| Other setup switches | One safety switch is on in the sandbox by design when it's off in production. Separately, the sandbox is missing several other switches: whether Clara is live, how the team is organized, how she offers to send an application link, her sign-off, and her feed of live listings. | Same on/off difference on that one safety switch. Also missing: which lead sources it should react to, and a few review-related switches. | Yes. Without the "how the team is organized" setting, a reply won't offer other homes in the portfolio, and the sign-off and application link differ. | Copy the real setup switches, except the ones that turn things on for real customers, and say so plainly in the report. |
| Phone line and shared inbox | Production has a real phone number and a real shared inbox. The sandbox has neither and uses a stand-in inbox. | Production has an office line, an emergency line and a shared team inbox. The sandbox has none. | Minor. It can change the contact info Clara mentions and a check for whether the team already has an email. | Note the difference in the readiness check; never copy the real phone line into a test sandbox. |
Two harness changes close every gap that changes an email: the provisioning step copies the real settings (except live switches, phone line, mailbox and AppFolio pointers), and the readiness check refuses to run when the sandbox drifts from production. Camellia gets its own sandbox so it stops inheriting Western Slope's answers.
An independent review made a point worth taking: the corpus score is a proxy. The real measure is how often a human approves Clara's draft as written, on live Western Slope prospects. We have that number, because every first email has been going through the review hold since 9/20.
Counted from production tonight, read-only. Since 9/20 there have been 22 held drafts at Western Slope: 11 approved, 10 rejected, 1 still pending. Today alone: 14 drafts, 5 approved, 8 rejected, 1 pending.
Split by which code wrote the draft, the picture changes completely:
| Drafts written on… | Approved as-is | What happened |
|---|---|---|
| the code running this morning 5 drafts, 11:28 AM to 12:39 PM | 1 of 5 — 20% | Three prospects' drafts were rejected and rewritten before one went out. |
| tonight's code 4 drafts recomposed 6:57–7:21 PM, after the greeting and gone-home fixes deployed | 4 of 4 — 100% | Elaine, Ben, Brody and Alexis all approved as written, 7:37–7:48 PM. Marissa's is still pending. |
Source: escalation-hold rows for org org_c42d9150…, entityType-createdAt-index, created on or after 2026-09-20. Read-only, nothing written.
The bar the review proposed, and we should adopt: 30 consecutive real held drafts at 90% or better approved as-is, with zero fabricated facts and zero stale dates, all measured after tonight's fixes. At today's rate — around 14 drafts a day — that is roughly two days of live volume. Wednesday night is plausible. Wednesday morning is not, because by morning we will have four.
Four real defects found tonight on live Western Slope drafts, with what the same email looks like after the fix. First names only; homes and prices are real.
Held draft, 5:37 PM tonight. A home-state sentence landed above the greeting, and the email template then added its own greeting on top.
| Before | After |
|---|---|
| Hi Alexis, The home at 2841 Morningside Court is open now. Hi Alexis, Thanks for reaching out about 2841 Morningside Court! … |
Hi Alexis, The home at 2841 Morningside Court is open now. Thanks for reaching out about 2841 Morningside Court! I'd love to help you get a closer look — I can offer a tour Thursday at 9:00 AM or Friday at 8:00 AM. … |
Proof: the 13 most recent real Western Slope drafts re-rendered through the old and new template. Doubled greetings before: 1. After: 0. The other 12 render byte-identical. (PR #11371)
The prospect wrote in from the listing page and asked for more information. Clara read them the specs they were already looking at.
| Before | After |
|---|---|
| The home at 2841 Morningside Court is a 4 bed, 2.5 bath single-family home in Orchard Mesa with 1,518 sq. ft., a 2-car garage, and a fenced-in backyard, listed at $2,300/month. It's available now… | Hi Sarah, Thanks for reaching out about 2841 Morningside Court — happy to help you find a time to come take a look, with an October 1st move-in in mind. I have Thursday at 9:00 AM or Friday at 8:00 AM open for a tour — would either of those work for you? |
Proof: 45-card replay of every Western Slope card that came from a listing site and named a home. Reciting before: 31. After: 0. The 14 that were already clean stayed clean, and none of them got worse. (PR #11344)
Clara said the home was no longer available, then offered tour times for it anyway.
| Before | After |
|---|---|
| The home at 239 S Elm Street, Unit 2 is no longer available. I don't have another open home to point you to at the property right now … In the meantime, if you'd still like to see the property, I have tour times open Thursday, September 24 at 9:00 AM or 3:00 PM … | The home at 239 S Elm St Unit 2 is no longer available. I do have two other homes open right now: 1250 Elm Avenue and 2841 Morningside Court. Happy to set up a tour of either one — I have Thursday at 9:00 AM or Friday at 8:00 AM … |
Proof: the 6 flagged replies replayed and re-judged. Sendable as-is before: 0. After: 4. (PR #11340)
The Zillow card asked Brody about his pets and he answered "no pets." Clara read that as the property's policy — and the property record says the opposite, cats and dogs welcome. A made-up fact about the home: the embarrassing class.
| Before | After |
|---|---|
| 2863 Florida Street is open now — a 3 bed, 2 bath home at 1435 sqft, $2,775/mo, and it doesn't take any pets so that's an easy fit for you. | The email does not go out. Clara now labels the card's pet answer as the renter's own, and any pet-policy claim the property record does not back is refused before sending. |
The draft was held and never sent. Fix is in review. (PR #11385)
Everything found tonight that is not yet merged. Nothing here is unknown; each line has a fix moving.
| What is still wrong | Kind | Where it stands |
|---|---|---|
| Pet policy invented from the prospect's own answer | Embarrassing | PR #11385, in review |
| A phone lead's home was recorded as the town, not the home | Embarrassing | PR #11393, in review |
| At Ridgway a listing address and the unit address disagree — 585 vs 565 Redcliff Cir | Embarrassing | Second PR in progress |
| Listing description prose beats the structured fields, so a home reads as "2.5 bath" when the record says otherwise | Embarrassing | #11397, PR in progress |
| A narrow window where the same email could send twice | Bland | #11370, logged |
| Bad drafts still become review holds instead of being refused | Bland | PR #11399, in review |
| A stale draft can still be approved, and a redraft does not update the record | Bland | #11387, logged |
| One "approved" reply in Slack was missed, at 5:15 PM today | Bland | Fix in progress |
| The review tab never listed held emails at all | Bland | PR in progress |
| A card naming four homes gets one record | Bland | #11380, logged |
| A lead can park with no email ever written — found tonight, it is why Marissa got nothing on Sunday | Embarrassing | #11379, logged |
Four of these are in the embarrassing class and not yet merged. That matters for the decision rule above.
| Option | What it is | When | Cost or risk |
|---|---|---|---|
| A. Go autonomous now | Remove the hold for all Western Slope first emails. | Not yet. We have four live drafts on the new code, not thirty. | A bad send reaches a real prospect before we see it. |
| B. Hold one more day | Clara keeps drafting, a human keeps pressing send, tonight's four fixes land, we count again Wednesday night. | If the live bar is not met. | Another day of Kat and Fede pressing send, slower replies to prospects. |
| C. Go autonomous, plus a same-day read of every send | Same as A, with a session reading every sent email that day and filing a bug for anything off. | The moment the live bar is met. | One session's attention for a day. |
Why the headline did not move. Send-as-is went 45% to 46% because tonight's fixes turned embarrassing replies into thin ones, not into good ones. Embarrassing failures fell 17 to 5. The next fix is substance — giving the reply something to say — not safety.
On "I'd rather ship it even if it's not 100%." Fair, and the reviewer agrees with the spirit. But 20% approved as-is this morning is not "not 100%" — four of every five drafts had to be rewritten by a person. Tonight's four-for-four after the fixes is the first real evidence the bar can be met. It is four, not thirty. One more day of live drafts settles it.
Fede asked four questions about the build as it stands today: do we name the right home, do we follow the policies, do we offer the calendar correctly, and do we escalate to a human accurately. This is a replay of real Western Slope cards through the same email composer production runs. Nothing was sent. Nothing was written to Western Slope.
Every Western Slope guest card created since 9/20 that exists: 19 cards (not 20–30 — 19 is what is there). 17 carry the renter's own words. For context we also replayed the 50 older Western Slope cards in the shared corpus (January to 9/19) that still have a PropFlow identity to compose against.
| Question | Pass | Fail | Not applicable |
|---|---|---|---|
| Right home | 12 | 0 | 7 |
| Policies grounded | 2 | 3 | 14 |
| Calendar offered correctly | 19 | 0 | 0 |
| Escalates to a human accurately | 7 | 12 | 0 |
A property manager would send 5 of the 19 exactly as written.
| Question | Pass | Fail | Not applicable |
|---|---|---|---|
| Right home | 25 | 22 | 3 |
| Policies grounded | 3 | 7 | 40 |
| Calendar offered correctly | 49 | 0 | 1 |
| Escalates to a human accurately | 3 | 47 | 0 |
The older set is much worse on the right home, and part of that is age: those cards point at homes that have since been rented, so the composer falls back to whatever is open now. It is not a clean read on today's behaviour, which is why the 19 recent cards are the number to trust.
Yes, when the card names one. On the 19 recent cards there were zero wrong-home drafts. 12 named the right home; the other 7 cards named no home at all (all of them phone or text leads where the caller never gave an address). Of those 7, the draft spoke about the portfolio generally and asked which home in 6 cases, and asserted one specific home the caller never mentioned in 1 case. That one is the bug in this dimension.
Not reliably, and the guard that is supposed to catch it did not fire once. 3 of the 19 drafts stated a policy or price fact that is not in the property's knowledge record. The blocking guard named unsupported-policy-claim fired on 0 of 69 drafts across both sets. Examples of what got through, in the drafts' own words:
This is the most serious finding on the page. It is a correctness bug, not a feature, so it is fixed and shipped, not gated.
Nothing offered was wrong, because nothing was offered. Across all 69 drafts the composer named zero concrete tour slots. It asks the renter what days work instead. No past times, no times outside hours, no tour of a home that is gone. That is the deployed behaviour and it is safe, but it does not yet book a requested time when that time is open — that is #11414, which has not merged. Until it does, a renter who writes “can I see it Thursday at 5?” gets a question back instead of a booking. The replay also logged the tour calendar refusing to bind for these properties, so even the lookup is not wired end to end yet.
Calendar, checked in production (11:00 AM MT): Production is fine. Every Western Slope first email composed since the evening of 9/23 (8 of 8) names real days and clock times, the compose logs show 4 open slots read on each of the three properties in the log window, and the live reply loop books real tours into the leasing agent's Outlook calendar. The zero-slot readings came from the replay targets: the sandbox org and the Willows test org have no tour-shower person attached, so the calendar resolver refuses there. That is a sandbox setup item for the redrive, not a go-live blocker. Two real bugs found on the way are filed as #11717 (replays write the calendar alert marker into production) and #11718 (calendar hook credential kind is never checked).
No. The composer held nothing — 0 of 69 drafts. Every draft with a fabricated address, an invented rent or an ungrounded policy line would have gone out on its own. It also missed the two clearest human-handoff cases in the recent set: a caller who explicitly asked for a live person and was frustrated about repeated calls, and a card that is an explicit staff handoff. Both drafted a cheerful reply instead.
The important caveat: in production these 19 did not reach anyone, because Western Slope's review-before-send gate is on and a human read each one. The engine's own judgement is what failed here; the human gate is what caught it. That is exactly the gate this page is about turning off, and on this evidence it should stay on.
Since 9/20, 22 drafts were held for a human: 11 approved as written, 10 rejected, 1 still pending. Two things the raw count hides: those 22 holds cover only 14 distinct cards (several were recomposed and re-held), and 2 of the 10 rejections were not about draft quality at all — they were the 9/23 lead-event flood, where an old inquiry resurfaced as new.
Re-composing each one on today's build and asking whether a human would now send it as written:
| Human verdict then | Clean on today's build | Still not clean | Card too old to replay |
|---|---|---|---|
| Approved as written (11) | 3 | 7 | 1 |
| Rejected (10) | 5 | 1 | 4 |
| Pending (1) | 0 | 1 | 0 |
Before: 11 of 22 send-as-is. After: 8 of 22, with 5 that cannot be replayed because their cards predate 9/20. Read that carefully — it is not a straight improvement. The build genuinely fixed most of what humans rejected (5 of the 6 replayable rejections are clean now). But it also regressed 7 drafts that a human had already approved. The net is flat-to-worse, and the movement is in both directions.
Every grader was tested with a planted violation to prove it can go red. All four caught their own defect:
| Planted defect | Home | Policy | Calendar | Escalation |
|---|---|---|---|---|
| Wrong home asserted | FAIL | — | pass | FAIL |
| Ungrounded policy claims | pass | FAIL | — | FAIL |
| Tour time in the past | pass | FAIL | FAIL | FAIL |
| Blocking defect, not held | — | FAIL | pass | FAIL |
The graders bleed into each other a little — a planted past tour time also trips the policy grader — so treat the per-dimension counts as directional, not precise to the row.
b4e3414f, live 7:18 AM today, one commit behind the tip of main.This is a replay, not a live test, and it grades drafts rather than outcomes — no renter read any of these words, so nothing here says whether a real person would have toured or leased. The recent population is 19 cards, which is small enough that one bad card moves a percentage point several points; the 50-card older set is bigger but its homes have since been rented, so its wrong-home count overstates today's problem. The grader is a language model reading the same knowledge record the drafting model had, so it can only catch a claim that contradicts or is absent from that record — if the knowledge record itself is wrong, a draft that faithfully repeats the error grades as a pass. The four planted controls show each grader can go red, but four cases is not a calibration; there is no measured false-positive rate against human labels, and the one place we do compare to humans, the 22 holds, disagrees in both directions. Five of the labelled holds could not be replayed at all because their cards predate the window. The calendar dimension is close to vacuous: every draft passed because no draft offered a time, so this run says nothing about whether booking will be correct once #11414 lands. And the escalation result measures the composer alone — production's human review gate sat in front of all of this and caught what the engine missed.
| Item | Where |
|---|---|
| Tuesday 5 PM replay, 205 cards with judge verdicts | ~/.claude/guest-card-corpus/runs/2026-09-23/ws-judged-with-messages.jsonl |
| 7 PM rerun on tonight's code, in progress | ~/.claude/guest-card-corpus/runs/2026-09-23/ws-composed-7pm.jsonl |
| Live approve-as-is counts | Escalation-hold rows, org org_c42d9150…, since 2026-09-20 (read-only) |
| One greeting per email | #11371, merged |
| Stop reciting the listing back | #11344, merged |
| No tour of a gone home | #11340, merged |
| No invented pet policy | #11385, in review |
| Readiness harness | bin/guest-card-corpus run --org org_western_slope, then judge |
Why this page exists separately: Fede asked for a very simple document to wake up to. The Western Slope issue list is the long tracker and links here.