ADR-0073 — Topic + Subtopic: one conversation categorization

Status: Accepted — implemented 2026-07-01 (PR #2796; prod backfill applied). Amended 2026-07-09: the voice CallTopic keyword classifier + fold table are RETIRED (see §"Amendment — the fold is retired" below); the shared maestro classifier (ADR-0084) is the one topic brain for every channel. Date: 2026-07-01 (amended 2026-07-09) Amends: ADR-0055 (conversation message topic taxonomy) Amended by: ADR-0095 (2026-07-15) — subtopic gains a per-topic vocabulary registry (SUBTOPIC_VOCAB) + a message-level field; the none catch-all's dispositions live under it.

Implemented 2026-07-01. Phases 0–4 shipped in PR #2796 (squash 42787e53b, merged 2026-07-01). The prod backfill was applied (279 rows, runId backfill_topic_subtopic_1782942452025_4cd59319; a re-run dry-run verified "Nothing to backfill"). The pre-implementation "Draft for review — no code ships until approved" banner is retired. The Conversation.issueType field-drop was executed 2026-07-02 (inventory + record: docs/planning/issuetype-field-drop-inventory.md); reintroduction is forbidden by conversation-issuetype-drop.drift.test.ts. Implementation plan (historical): docs/planning/topic-subtopic-taxonomy.md.

Context

A conversation currently carries two category fields that overlap, use different vocabularies, and are populated on different channels:

• topics (ADR-0055, values from OutboundRegardingType) — the domain buckets: maintenance · leasing · tour · renewal · turnover · billing · lease · resident_inquiry · vendor · tenant_confirmation · emergency. Multi-tag, derived from message text by the message-topic-classifier (inline on save
  • the nightly conversation-topics cron). Denormalized onto the conversation meta row so the message-less list can render a Topic column. tenant_confirmation and emergency are the structural (non-classifier) topics — event-stamped and sticky, never text-classified (see ADR-0055 §"Structural topics").
• Conversation.issueType (string | null) — overloaded, doing two unrelated jobs:
  1. the maintenance sub-type — VALID_ISSUE_TYPES (the 13 troubleshooting playbooks: running_toilet, ac_not_cooling, …). This is the role ADR-0055 §"existing vocabularies" explicitly reserved for it: "too granular for a topic … it stays a maintenance-intake sub-classification, NOT a topic."
  2. the voice call-topic — the CallTopic enum (renewal · rent_payment · lease_question · tour_inquiry · noise_complaint · lockout · move_in_out · parking · general_inquiry), written by classifyCallTopic in call-ended.
  3. the WO trade — a WorkOrderCategory (plumbing · hvac · electrical · appliance · general), written by the WO-create paths (call-ended, handle-create-work-order) when no troubleshooting playbook ran. This is a coarser maintenance detail; under topic + subtopic it is conveyed by topic=maintenance (the trade lives on the WorkOrder), so subtopic holds the FINE-grained playbook key from (1), not the trade.

Observed on production (Camellia, last 24h): text channels get topics, voice gets issueType — because the ADR-0055 message classifier never runs on voice transcripts. So:

Neither field alone answers "what is this conversation about," and issueType carries two different vocabularies at two different levels. This is exactly the parallel-implementation smell the repo's one-source-of-truth rule forbids.

Decision

A two-level model, both anchored on the ADR-0055 vocabulary:

• topic — the universal domain bucket (the ADR-0055 taxonomy). Stays multi-tag (a thread can legitimately be [leasing, maintenance]), and is now stamped on every channel, including voice.
• subtopic — optional, nullable, granular detail scoped under a topic. Today only maintenance has one: topic=maintenance → subtopic=running_toilet (VALID_ISSUE_TYPES). Extensible per-topic later; never forced on a topic that has no meaningful subdivision (a tour has no subtopic).

Conversation.issueType is retired as a category carrier. Its two meanings split cleanly:

• maintenance-playbook value → subtopic (its intended, narrow role).
• voice call-topic value → folded up into topic via the mapping below (2026-07-09: the fold — and the keyword classifier feeding it — is retired; voice free-text is classified by the shared maestro via ADR-0084. See the Amendment below.)

Fold: CallTopic (voice) → topic — RETIRED 2026-07-09 (see the Amendment below)

The table below is the historical fold. It no longer exists in code — classifyCallTopic, CALL_TOPIC_TO_TOPIC, and foldCallTopicToTopic were deleted; voice transcript rows now stay untagged at call-end and are classified by the shared maestro via the ADR-0084 stamp workflow.

Rejected alternative — type / subtype

type/subtype collides with the existing issueType, OutboundRegardingType, and the domain entity types, and abandons the ADR-0055 "topic" vocabulary the code already references. "Topic → Subtopic" reads cleanly for PMs ("Topic: Maintenance · Subtopic: Running toilet") and engineers alike.

Consequences

• One topic answer everywhere: the conversations list + drawer, the detail badge, global search, and the daily conversations report all read topic (+ subtopic where the maintenance detail is wanted).
• Maintenance granularity lives in exactly one place (subtopic); no field does double duty.
• Voice conversations finally carry topics.
• This is a migration that deletes Conversation.issueType — not a third parallel field. A backfill runs before the type flip; a drift guard forbids reintroducing issueType as a conversation field.
• Untouched: Playbook.issueType (the playbook key on the Playbook entity / getPlaybook / the get_troubleshooting_steps tool) is a different field and stays exactly as-is.

Decisions & open questions

Resolved (2026-07-01):

1. move_in_out → turnover. A move (in or out) most often lands the thread in the turnover domain; when the CallTopic value is all we have we fold to turnover rather than leasing. (A future transcript-direction signal could split move-in → leasing / move-out → turnover, but that is not required for the fold.) STRUCK 2026-07-09 — this decision (and the whole fold) is retired; see the Amendment below. The prospect saying "I would like to move in next month" is a leasing call, and no fold row could ever have been right for both directions of a "move".
2. general_inquiry → untagged. Per ADR-0055, the classifier abstains over over-tagging — general_inquiry carries no domain, so it folds to no topic. (Historical — the abstain posture survives, but as "leave the row untagged for the maestro", not as a fold entry.)

Amendment (2026-07-09) — the voice keyword classifier + fold are retired

What changed. src/lib/integrations/voice/classify-call-topic.ts (the 9-word regex CallTopic classifier), CALL_TOPIC_TO_TOPIC, and foldCallTopicToTopic are deleted. The voice call-ended handler no longer stamps any text-derived topic on transcript rows. The only synchronous stamps that survive are STRUCTURAL ones — as first shipped, exactly one: when the post-call sweep minted a work order, the WO-report segment rows get bare maintenance (no regardingId, so the post-save back-stamp still upgrades the DDB rows to the full WO tie — its condition is attribute_not_exists(regardingId)). A minted WO is a structural fact, not a text guess. A second structural stamp joined it on 2026-07-31 — the recognized vendor call-back; see that amendment below.

Segment-scoped, not every-row (2026-07-10 correction). The amendment as first shipped stamped maintenance on EVERY untagged transcript row of a WO-minting call — which re-created the exact lockout this amendment exists to kill, from the other direction: with every row tagged, the ADR-0084 stamp workflow's untagged-count guard made the maestro a no-op, so the whole call resolved to [maintenance] and any second intent (a renewal ask, a billing question) was invisible. The stamp is now scoped to the same rows the post-save backstampWorkOrderReportSegment ties to the WO — computed by the same pure selector (selectReportSegment, via applyReportSegmentTopicStamp in wo-report-backstamp.ts), so the in-memory topic set and the DDB tie set are equal by construction. Rows outside the segment (older than a prior different-entity boundary, or beyond REPORT_SEGMENT_CAP) stay untagged for the maestro. Because the stamp rides real message rows, every later full save in the handler (recap, transfer) re-derives maintenance onto the meta row from those same in-memory objects — maintenance is deliberately NOT in STICKY_TOPICS and there is no meta-row union: it ages out with the 14-day message window like any other message-borne topic.

Known limit (open, documented — do not paper over). selectReportSegment is structural: it walks back to the first row tied to a DIFFERENT entity. A fresh per-call voice conversation has no such boundary, so the segment degenerates to the whole trailing transcript (≤ cap = 30 tenant/assistant rows) — and the post-save back-stamp ties those same rows in DDB regardless. A second intent voiced inside the same short call ("my dishwasher is broken, and I want to renew my lease") is therefore still swept into the WO's segment and never reaches the maestro. Fixing that requires the intake classifier to emit per-issue message refs (a classifier prompt/output-schema change, deliberately out of scope for this correction — it trips the promptfoo-eval gate and its abstain path was the source of a prior failed attempt). What IS fixed: rows beyond the cap, rows behind a prior entity boundary (e.g. an SMS-seeded thread), and — the blocker-2 regression — the recap/transfer re-saves can no longer drop maintenance from Conversation.topics, because it is carried by the report-segment rows themselves.

The bug that forced this. The CallTopic vocabulary had NO leasing value. A prospect saying "I would like to move in next month" matched the move_in_out regex and folded to turnover. Because applyTopicTags never overwrites an existing tag, and the stamp workflow's untagged-count guard only spends an LLM call when untagged rows exist, that regex guess was PERMANENT — it locked the real LLM maestro out of the conversation. ADR-0084 made call-ended signal the per-conversation stamp workflow (maestro classifies within ~2 min), but the regex stamped first, so the workflow found nothing untagged and returned without classifying.

Why not the transcript-direction signal? Resolved decision 1 anticipated a "future transcript-direction signal" to split move-in → leasing / move-out → turnover. That would have patched exactly one fold row and left a keyword regex as the sole topic authority on voice — every other fold row (and every future vocabulary gap) would still pre-empt the maestro. The wrong layer was doing the classifying; refining its guess was the wrong fix.

The one-brain principle. ADR-0054 Slice 5 already established it for maintenance intake — quoting the voice-postcall-intake.ts header: "ONE-BRAIN means one CLASSIFIER, not one transport entry." Voice may enter through its own transport (a dead call has no reply to render), but the decision runs on the same brain as text. The same rule now holds for topics: voice transcript rows are left untagged at call-end, and the scheduleConversationTopicStamp signal already fired by the handler (ADR-0084) has the shared maestro classify them — the identical classifier, vocabulary, and abstain posture every text channel gets. Untagged-at-call-end is the correct and complete behavior for rows we hold no structural fact about, not a gap. (2026-07-31: "untagged" was written when the WO-report segment was the only structural exception. It reads the same way with two — a row is untagged unless something structural, never a model reading words, already answers what the call was.)

Trade-offs accepted. The maestro path is an LLM call per voice call where the regex was free, and it is non-deterministic where the regex was deterministic. Both are the price of correctness: the regex's determinism was deterministically wrong for any vocabulary it lacked, and the maestro's abstain discipline (ADR-0055 — prefer no tag over an over-tag) is pinned by its own evals.

Guards. conversation-topic-taxonomy.drift.test.ts now asserts the classifier file does not exist and topic-labels.ts contains none of the retired symbols; api-routes-voice.test.ts pins the regression transcript ("I would like to move in next month" must never produce turnover), the surviving structural maintenance stamp (segment-scoped, type-only, ordering stamp → saveNewMessages → saveConversation → back-stamp, and set-equality between the in-memory stamp and the back-stamp's refs), and the cap-overflow mixed-intent case (rows beyond the segment stay untagged for the maestro); wo-report-backstamp.test.ts pins applyReportSegmentTopicStamp (type-only, boundary-respecting, never-overwrite, walk-idempotent); voice-wo-mixed-intent-topics.test.ts proves a renewal turn outside the segment survives the REAL maestro classification path and that the recap re-derivation keeps maintenance.

Amendment (2026-07-31) — a second structural voice stamp: the recognized vendor call-back

What changed. applyVendorCallbackTopicStamp (src/lib/domain/messaging/vendor-callback-topic-stamp.ts) stamps vendor on a recognized vendor call-back's transcript rows at call-end. The amendment above says the WO-report segment is the only synchronous structural stamp — as of this change there are two.

The incident (Trello LnRbYW7L). A blinds vendor rang back to move an install and chase an order approval, over a bad truck connection. A recognized call-back is deliberately kept OUT of the new-work-order sweep (ADR-0111 §5 — so it can never file a second job for the one it is calling about), and that sweep was the only structural voice stamp, so the entire transcript reached the maestro untagged. The maestro read the call's dead-air stretches as the whole call: every row none. An all-none thread earns a disposition (ghosted = "Dead air"), and every disposition is hidden by default on /conversations. A real vendor call vanished from the main page.

Why a stamp and not a better classifier. Recognition already happened, with evidence — their number matched a job we dialed about, or the outcome handler tied the call to a bound job. That is the same class of fact the WO-report stamp rests on: something the handler KNOWS, not something a model read out of words. Once vendor is on the rows, three existing invariants finish the job with no new machinery: applyTopicTags never overwrites a real topic; evictNoneWhenRealTopicPresent keeps the derived set from ever collapsing to ['none'] (so no ghosted/spam disposition can be written); and searchableTopics evicts none on read, so no -none:* hide group matches the row.

Whole-batch, NOT segment-scoped — and that difference is the cost. The 2026-07-10 correction above exists precisely to stop an every-row stamp from making the ADR-0084 stamp workflow's untagged-count guard a no-op, and this stamp has exactly that shape: it tags every dialogue row THIS call minted. It is accepted here, not overlooked, because the batch is one short call about one thing (vendor is true of all of it) and subtopicEnrichableCount still lets vendor subtopics land — but the consequence is real and identical to the WO path's known limit: a second intent voiced inside the same call can never surface as its own topic. Scoping tighter needs classifier-emitted per-issue message refs, the same blocked prerequisite the WO path documents. Until then the trade is deliberate: a permanently narrow topic beats a call that is hidden from the page entirely.

Guardrails, each pinned by a test.

• Fresh-batch only. The stamp takes the rows THIS webhook minted (messages), never conversation.messages — which silently falls back to the thread's already-persisted PRIOR rows when a webhook mints none. Stamping those would retro-tag another call's transcript, read its speech as this call's, and flip the silent-hangup gate to false.
• Zero caller speech stands down. A call-back the vendor never spoke a word on is a silent hang-up, and ADR-0095's structural silent_hangup path owns it (it gates on "no real topic anywhere"). Blank-content rows are not speech.
• First-tag-wins. A row already holding a REAL topic keeps it; only the none sentinel — the absence of a provenance, not a provenance — is upgraded.
• Skipped when the call minted a NEW work order. That is fresh intake, and the WO tie is the stronger fact.
• Strong recognition only (narrowed in review, 2026-07-31). The gate is ringTimeBoundJob || vendor_callback_exact_dial || boundJobConsumedTheCall. It is deliberately NARROWER than the intake suppression that sits ~170 lines above it, because the suppression is reversible paperwork and this stamp is permanent. Refused: the weak mid-call lane (a company name the caller asserted, matched on a distinctive-token subset), and the ring-time membership / pms_contact_phone lanes — vendor_callback: 'true' is set on matchedBy === 'exact_dial' || !isKnownCaller, so those raise it for any caller we merely failed to recognize. Without the narrowing, an unknown caller dialing from a number on some vendor's contact record, whose call mints no work order, would be stamped Vendor forever with the maestro locked out.

Guards. api-routes-voice.test.ts pins the stamp on both strong ring-time lanes and on the earned mid-call lane, the weak-lane refusals (ring-time and mid-call), the WO-minting stand-down, the ordinary caller, and the empty-transcript scope contract (prior rows untouched, silent_hangup still written); it also pins that personalization emits vendor_callback_exact_dial for the strong lane and never for a weak one. vendor-callback-not-dead-air.test.ts pins the helper at the unit seam and carries a mixed thread through the real planConversationStamp.

Still open (does NOT block Phase 1):

3. Multi-tag + subtopic coupling. Since topic is a set, subtopic attaches to the maintenance tag specifically, not the whole conversation. Ship subtopic: string | null (single) for now (only maintenance has one); revisit subtopics: Record<topic, string> only if a second topic ever gains a subtopic vocabulary.