0084 — Inline conversation-topic stamping: signal on turn-settle + 15-minute reconciler

Status: Accepted — premise partially corrected by ADR-0088 (2026-07-14); reconciler half DELETED by ADR-0094 (2026-07-16)
Date: 2026-07-09
Deciders: Jose (Gera)

Debounce cut (2026-08-28): the workflow's idle debounce dropped from 2 min (hard cap 10) to 10s (hard cap 60s) — owner call: a multi-minute artificial hold on a customer-visible "Classifying…" row bought batching savings on a per-classify LLM call cheap enough that the savings didn't justify the wait ("I'd rather be wrong than waiting"). See conversation-topic-stamp-workflow.ts's DEFAULT_DEBOUNCE_MS comment. BLANK_TOPIC_GRACE_MS (the blank-topic regression alert's grace window) is unchanged at 45 min — a PR #6479 review finding established the debounce was never its dominant term (that predicate gates on conversation startedAt, i.e. birth, not the last settled turn — a long voice call and the retry ladder's ~32-min worst case are what actually set the floor; see classification-observability.ts's BLANK_TOPIC_GRACE_MS comment).

Deletion (2026-07-16, see ADR-0094): the 15-minute conversation-topics-reconciler sweep this ADR introduced is deleted — workflow, activity, schedule, upsert script, domain sweep, and the reconcilerCaughtMissed instrument. The ADR-0088 event stream signals the inline conversationTopicStampWorkflow per settled turn on every channel (verified live in prod), making the sweep pure redundancy. The inline stamp path in this ADR is unchanged and remains the SOLE path. The reconciler sections below are retained as history.

Correction (2026-07-14, see ADR-0088): this ADR's framing that "the inbound-processor Lambda cannot signal, so the 15-minute reconciler is the primary coverage for Lambda-origin (SMS/email) conversations" is no longer true. As of ADR-0068 (2026-06-24) the Lambda carries TEMPORAL_API_KEY and isTemporalConfigured() returns true in all prod runtimes, so the inline conversationTopicStampWorkflow signal fires from the Lambda too. The reconciler is now a dropped-signal backstop, not the primary SMS/email path. The classify-and-stamp logic in this ADR is unchanged and correct; only the "who triggers it" premise moved. ADR-0088 generalizes this ADR's per-consumer inline trigger into a single durable conversation.turn_settled event stream (one emitter, one bridge-signaler) shared by topic-stamp, grade, maintenance, and future consumers, and moves the Temporal dial off the reply hot path. Read ADR-0088 before extending the trigger path.

Context

A conversation's Topic chips (ADR-0055) come from message-level regardingType stamps. Deterministic writers — outbound provenance and entity ties for maintenance, leasing, tour, renewal — stamp synchronously at write time. Free-text inbound (a tenant's "yes I'll renew", "the sink's still leaking") carries no deterministic topic and is classified by the maestro (classifyThreadTopics), which until now ran ONLY inside the nightly conversation-topics-nightly Temporal Schedule (PR #2382, an explicitly labeled interim bridge: "retire when every workflow stamps topics inline").

The cost of nightly batch: an inbound reply sits with a blank Topic for up to ~24h before the sweep tags it. On the standardized conversation surfaces (PR #3414 — one ConversationsLog, scoped pages filtering on topics) that reads as "data missing from the scoped page," the exact class of gap the standardization set out to eliminate.

Constraint discovered while scoping: the SMS/email inbound-processor Lambda — where most inbound agent turns settle — has NO Temporal creds. A signalWithStart from it dials the localhost:7233 default and times out on every message (the 2026-06-14 latency incident; the maintenance pipeline's trigger carries the same gate and the same known gap). So a pure signal-on-settle design silently skips the largest inbound source.

Decision

Two paths, one classify+stamp module (conversation-topic-stamping.ts — the same classifyAndStampConversation both consume, so they cannot diverge):

Inline signal (conversationTopicStampWorkflow). When an inbound turn settles on a Temporal-configured runtime (Vercel: voice call-ended — the majority channel — plus simulator/pipeline-lab/dashboard turns), the trigger scheduleConversationTopicStamp(conversationId) signalWithStarts a per-conversation workflow (conversation-topics-<conversationId>, USE_EXISTING). The workflow debounces 2 minutes of idle (hard cap 10) so a burst of texts classifies once, then runs stampConversationTopicsActivity — one LLM classify, stamps, re-derives Conversation.topics. Mirrors the maintenance trigger exactly: Temporal-configured gate, best-effort (log + swallow), lazy client import. NOTE (ADR-0073 amendment, corrected 2026-07-10): the workflow only spends its LLM call when untagged classifiable rows exist, so synchronous stamps upstream must stay MINIMAL — the voice call-ended handler's structural maintenance stamp is scoped to the WO-report segment (applyReportSegmentTopicStamp), never the whole transcript, precisely so this maestro path still sees the remaining rows. AMENDED 2026-07-31 (ADR-0073 amendment, "a second structural voice stamp"): a SECOND synchronous structural stamp now exists — vendor on a recognized vendor call-back (applyVendorCallbackTopicStamp) — and it is whole-batch, not segment-scoped, so on those calls the untagged-count guard does make this maestro path a no-op. That is a knowingly accepted cost, not a violation of the MINIMAL rule: such a call-back is otherwise stamped by nothing at all (it is held out of the work-order sweep by ADR-0111 §5), and the failure it fixes is worse than a lost second intent — an all-none thread earns a ghosted disposition and disappears from /conversations. The MINIMAL rule still binds everything else: the stamp fires only on STRONG structural recognition (a ring-time bound job, an exact-dial match, or the outcome handler's own verdict), never on a weak or self-claimed one, so the maestro keeps every call we do not already hold a fact about.
15-minute reconciler (conversation-topics-reconciler). The existing sweep workflow re-shaped: nightly 08:00 UTC / 26h window → every 15 minutes / 1h window, renamed with the cadence change so the schedule id stops describing a cadence it no longer has. It covers (a) Lambda-origin inbound that cannot signal, (b) any dropped signal. Overlap is free — the maestro never overwrites and nothing-untagged conversations are skipped before any LLM call, so total classify volume is unchanged from nightly (work is proportional to new untagged messages, not sweep frequency).

Upsert paused-semantics fix (bug found in review of the reshape): the old upsertConversationTopicsSchedule defaulted paused: true on UPDATE too — a routine spec upsert would silently pause a live schedule. Now: unset paused preserves the live state on update and lands paused on first create.

Skip semantics are shared: test properties (Property.isTest) are excluded on both paths (the census showed appfolio-45 is 76% of untagged — by design); a conversation with nothing untagged never invokes the maestro.

Consequences

Positive: Topic chips populate in ~2 min for Vercel-origin turns (voice = 57% of volume) and ≤15 min for everything else — from ≤24h. The scoped conversation pages stop under-reporting fresh threads. One domain module still owns classify→stamp→derive.
Cost: one more workflow type on the existing propflow-conversation-topics queue (no infra change; same Fargate worker). The reconciler's per-run overhead (getProperties + per-property meta query × 96/day) is trivial at current fleet size; the pre-existing scale-watch (server-side lastMessageAt filtering before ~50 properties) carries over unchanged.
Accepted race: a signal landing after the debounce elapsed but before the run completes is absorbed without re-classifying (USE_EXISTING joins a finishing run). The reconciler re-covers within ≤15 min; the maestro never overwrites, so the worst case is a briefly-late chip.
Retirement path: the reconciler retires when the inbound-processor Lambda gains Temporal creds and every turn signals inline — the SAME tracked gap as the maintenance SMS-origin signal. Until then it is load-bearing, not vestigial. Done — retired 2026-07-16 (ADR-0094): the Lambda signals via the ADR-0088 stream, and the reconciler was deleted outright.

Alternatives considered

Keep nightly, add signals only. Rejected: Lambda-origin SMS/email (the bulk of free-text inbound) would stay next-day — the user-visible problem survives for the largest channel.
Reuse the inbound router's intent as the topic (no LLM). Rejected as the sole mechanism: only tags turns that hit a domain flow; ambiguous replies — the actual gap — still need the maestro. Deterministic writers already cover the router-known cases.
Give the Lambda Temporal creds now. The real end-state, but an infra/security change (SSM creds, connection lifecycle in a frozen-container runtime) shared with the maintenance-signal gap — out of scope here, tracked separately.
Classify synchronously inside the agent turn. Rejected: adds LLM latency to Clara's reply path for a chip that tolerates minutes of lag.