0122 — Evidence-gated PM notifications: a claim to a PM binds to observed evidence, never to a phase or an extraction
- Status: Accepted (Fede, 2026-08-01) — implemented and prod-verified via #5274, #5288, #5291 (12/12 audited surfaces honest on the evidence-gate harness, up from 0/12)
- Date: 2026-07-31
- Decider: Fede — drafted from the 2026-07-31 false-countersign-notification investigation ("this tells me there might be a big architecture flaw in how we do reminders, so do a deep inspection and come up with a strong fix that addresses the roots"); approved 2026-08-01 with the direction to implement plus a harness proving the new behavior against the old
- Builds on: ADR-0104 (PM action reminders — this ADR adds the premise half of its fail-closed re-check), the ADR-0035 Person spine, the vacate lane's existing stated-vs-formal split (
signal-tenant-renewal-bridge.ts"Formal ≠ verbal") - Tactical containment already shipped: #5173 (countersign cadence requires signature evidence, replay-safe patch gate)
- Evidence: four-stream inspection 2026-07-31 — reminder-system map, 19-surface claim/evidence audit, intent-vocabulary audit, industry research. Key citations inline.
Context: the incident, and why it is a class
On 2026-07-31 a resident of a live customer property emailed: she wants to renew if rent drops by roughly 13% (a ~$1,150 → ~$1,000 ask). Within seven seconds PropFlow told the PM: "
The chain (full trace in the landed audit): Haiku extraction stamped intent_to_renew (the schema cannot express a conditional acceptance) → the signal bridge fired tenantIntent{kind:'renew'} (signal-tenant-renewal-bridge.ts:653-658) → one line in the renewal workflow OR'd that LLM belief with a PMS-verified signature:
// renewal-workflow.ts:1279
if (pms() === 'tenant_signed' || (opts.countersignOnRenewIntent && intent()?.kind === 'renew'))
→ resolveCountersignWait entered awaiting_countersign and started the renewal_countersign cadence, whose copy asserts a signature (pm-action-reminder-copy.ts:116-118).
This is not one bug. The 19-surface audit found the same defect on 11 surfaces across two domains:
| # | Surface | Claim | Actual evidence | Severity |
|---|---|---|---|---|
| 1 | renewal_countersign reminder |
"agreed to renew… just needs your signature" | LLM-extracted email intent | critical |
| 2 | Saga activity log (renewal.ts:2550) |
"Tenant signed — waiting for the lease to be countersigned" | internal phase alone | critical |
| 3 | countersign_timeout escalation (email+SMS, renewal-escalation-copy.ts:247-257) |
"agreed to renew, but the lease hasn't been countersigned" | may be intent-only path | high |
| 4–5, 7 | Vendor call outcome notices (notify-pm-outcome.ts:340-394) |
"{Vendor} is scheduled for {day/time}" / "moved off {old day}" | LLM extraction over ASR of a call; no confirmation, no calendar/PMS write | high |
| 6, 9 | Holdover office heads-up (renewal-holdover-notify.ts:98) |
"We've sent them a fresh renewal offer" | a workflow was started; it can still fail or hold | medium |
| 8 | application_review reminder |
"applied for Unit 204" | targetUnitId is the unit of interest (types.ts:9599-9601), not the applied-for unit; empty values already degrade safely to "unit unknown" |
medium |
| 10 | forwarded_question reminder |
verbatim-attributed question | LLM one-line summary | low |
| 11 | Move-out draft at inspection complete (fireMoveOutDraftAtInspectionComplete) |
"is moving out on {date}" asserted as fact | PropFlow's own record; the stale-date guard waves the draft through when the PMS has no date at all (inspection-complete-move-out-draft-hook.ts:123 if (!pmsObservedDate) return null) — zero PMS corroboration in exactly that case |
partial, low/medium |
And the mirror-image gap: the voice/SMS lane correctly refuses conditional acceptance (renewal-core.ts:63: "Conditional acceptance is NOT acceptance… escalate, don't accept") but handleRenewalEscalateImpl never signals the workflow — so the workflow keeps nudging a tenant who already made a counter-offer, then escalates for "no response." Email over-commits; voice under-records. Same missing concept. Meanwhile the board projects a counter-offering tenant to RENEWED (renewal-view.ts:796-816), which excludes them from churn risk (renewal-risk.ts:192) at exactly the moment they are negotiating.
The structural flaw, in one sentence: three things must agree — the trigger's evidence, the resolution predicate, and the copy's assertions — and the ADR-0104 engine only enforces one edge of that triangle (completion-evidence ↔ resolution predicate). There is no premise predicate: nothing ever verifies that the fact a notification asserts was true when the notification fired. The codebase already states the right principle on the vacate lane — formalNtv has a conservative parse bar because "a false positive fires move-out machinery"; the backstop refuses to fabricate signature evidence ("no clean signal… so we do NOT fabricate one") — but the renew lane got neither. The read models are already honest (board/detail gate on the PMS-verified tenantSignedNotifiedPmsRef); only the push notifications skip the gate. lease_countersign — double-confirmed e-sign parties before any email (l4-core.ts:1828-1835) — is the in-house exemplar of the standard.
Industry practice draws this exact line three times over: DocuSign/Dropbox Sign separate per-signer events from all_signed and document that you act only on the latter; Stripe's events are "thin" — the event schedules the message, a re-read of the object supplies the claim; sales pipelines keep "verbal commitment" as its own stage whose exit criterion is a counterparty artifact (a signed contract), never a system-side action.
Decision 1 — Evidence class is a typed, first-class property of every intent and event
Every tenant-decision signal carries its provenance:
TenantIntentPayloadgainsevidence?: 'pms_verified' | 'stated'and a third kind,'negotiating'. Replay contract: the union members are additive (every existing read is=== 'renew'/=== 'vacate');evidenceis optional on the wire and absent means'stated'at every read site — a signal recorded before this landed can never have been PMS-verified, so the fail-closed default is the historically-correct one. Slice 3 adds a pre-changetenantIntenthistory fixture to the replay corpus (ADR-0108 D5) to pin this.OperationalSignalgainsintentQualifier: 'unconditional' | 'conditional' | 'question'andintentCondition(free text, e.g. "rent reduced to $1,000"). The extractor prompt gets a conditional bar on the renew side symmetric to theformalNtvbar on the vacate side, plus eval fixtures for counter-offers (today the eval'sexpectedtype cannot even express one).- A conditional or question-qualified renew intent maps to
kind:'negotiating', neverkind:'renew'.
Decision 2 — Stated intent and verified signature are different workflow states
The load-bearing OR at renewal-workflow.ts:1279 is split:
pms() === 'tenant_signed'(the L4 ready-to-countersign feed, which checks per-party signatures) is the only entry toawaiting_countersignand the only trigger of the countersign cadence. (#5173 already enforces the cadence half.)- A bare
kind:'renew'intent enters a newcommitment_statedposture: tenant outreach stops (they said yes — don't nudge them), the board projects "Says they'll renew — awaiting signature," and the tenant-facing nudge ladder switches to sign-reminder copy. It is the sales-pipeline "verbal commitment" stage: real, valuable, and not renewed. kind:'negotiating'routes to PM review (the same posture voice'srenewal_escalatetakes), projectsRENEWAL_NEGOTIATINGon the board with plain-English copy ("Negotiating terms" — an enum must never be the PM-facing string), and counts toward churn risk.handleRenewalEscalateImplfires this signal too, closing the voice under-record gap.
Splitting the OR removes its right-hand side entirely: the countersignOnRenewIntent option is deleted from resolveTerminalLadder's opts, along with its four call-site arguments and its comment block — no permanently-false parameter left as scaffolding.
This mirrors the vacate lane's existing, proven split (INTENT_TO_VACATE stated vs NTV_RECEIVED formal).
Decision 3 — Every PM-action kind gets a premise predicate, enforced fail-closed
PmActionKindHandler grows a premise predicate — the mirror of its existing resolution predicate: what observed fact must be true for this action to be askable at all (renewal_countersign: a tenant signature observed in the PMS; application_review: an application on file for the named unit; …). It is re-checked fail-closed at cadence start and before every send, exactly like the existing pre-send resolution re-check. An unverifiable premise means no send — the same "we do not fabricate evidence" rule the backstop already applies to lease_countersign. Predicate-design hazard to encode: coalescing readers make absent fields look present — e.g. the application_review premise must treat targetUnitId === '' as absent, not as a unit on file (the type's reader contract coalesces missing to '').
Decision 4 — Copy binds to re-read evidence, and stated evidence renders as attributed speech
- Every factual sentence in a PM-facing template binds to a typed field on the authoritative record re-read at render time — never to the triggering event or a workflow phase (the Stripe thin-event rule).
- Evidence class selects the copy register:
pms_verifiedmay assert ("The tenant signed");statedmust attribute ("Their email says they intend to renew — nothing is signed yet"); LLM-extracted content is always attributed, never verbatim-quoted as the counterparty's words. - A regression suite pins claim↔evidence for every PM-facing template (today no test anywhere asserts copy matches trigger evidence; the replay corpus covers only
forwarded_question).
Consequences
- The stated-intent-as-signature class dies at three independent layers (typed intent, phase split, premise predicate) instead of one string edit.
- Counter-offers become visible revenue events: routed to a human, on the board, in churn risk — instead of being silently marked won.
- The vendor-call surface (#4–5, 7) adopts the same register rule: "The vendor said Tuesday works" (attributed) until a calendar/PMS write exists.
- Cost: extractor prompt + eval additions; one workflow patch gate (
patched()— same discipline as #5173); premise predicates for four kinds; copy edits on 11 surfaces.
Rollout (by blast radius)
- Slice 1 (shipped, #5173): countersign cadence + timeout copy gated on signature evidence.
- Slice 2: activity-log copy (
renewal.ts:2550) +countersign_timeoutescalation fan-out readpmsoff the same input object they already receive;commitment_statedboard projection. - Slice 3:
kind:'negotiating'— extractor qualifier, bridge mapping,handleRenewalEscalateImplsignal,RENEWAL_NEGOTIATINGprojection + churn-risk inclusion. Same slice ships theevidencefield (absent ⇒'stated'at every read site) and the pre-changetenantIntenthistory fixture in the replay corpus — ADR-0108 D5 makes that fixture a merge gate. - Slice 4: premise predicates on all four PM-action kinds; copy-register sweep of the remaining audited surfaces (vendor calls, holdover, application_review, forwarded_question); claim↔evidence regression suite.
Adjacent defects found by the same audit (filed separately, not gated on this ADR)
lease_countersignresolve signal keyed onleaseIdwhile the cadence anchors onesignDocId— the resolve never lands; cadences close only by budget/expiry andresolvedAtrecordsnow().- Collections chase reuses
forwarded_questioncopy ("Clara passed along a question") for a past-due-balance escalation. - Renewal bridge stamps
openedAt = now()with date-granular comparison — same-day races possible. src/lib/domain/messaging/__tests__/conversation-writer-ensure.test.tscarries a real tenant name + unit as a fixture (almost certainly minted off a real row) — anonymize per the no-PII rule.fireMoveOutDraftForEmailNtvcurrently has no production caller; wiring it as-is would clone this failure class into turnover (its own header calls the date an "email-derived estimate").