ADR-0027 — Three-layer architecture: Catalog / Events-Values / Spine; Canonical vs Derived

• Status: Accepted
• Date: 2026-05-16
• Deciders: Gera (Jose) — author
• Supersedes: none
• Extends: ADR-0018 (Person identity model), ADR-0019 (Organization model), ADR-0020 (Person as universal human spine — proposed). Codifies the architectural taxonomy that ADR-0018 began and Phase 5a operationalized.

Context

The data-model migration that ran 2026-05-02 → 2026-05-15 (Phase 5a closure) collapsed the legacy Tenant/Prospect/Vendor single-phone-keyed topology onto a Person + IdentityClaim + role-relation spine. The migration's load-bearing rule was "100% legacy elimination — only one source of truth" (Jose's GOLDEN/HARD rule, 2026-05-15). Phase 5a closed clean: 13 forbidden compat/shim/shadow patterns CI-pinned dead via src/__tests__/legacy-field-elimination.drift.test.ts.

The full-phase audit filed 2026-05-16 (PR #982) surfaced the next architectural question. Two cross-cutting registries — the Tool Catalog (src/lib/tools/, ~50 entries) and the Metric Catalog (src/lib/data/metric-catalog.ts, hundreds of keys) — sit outside the spine. The catalog defines what's reportable; the corresponding MetricSnapshot DDB rows and AgentTraceStep rows record what actually happened. The question came up: should MetricSnapshot carry a byPersonId: Record<string, number> to answer "who called?" Should ToolInvocation be a first-class entity to make tool calls queryable?

Both proposals would reintroduce the exact dual-write apparatus Phase 5a tore out. The answer is to recognize three distinct classes of data and name + scope them precisely so they can't be conflated again.

The repository today has ~70 interfaces in src/lib/data/types.ts + 2 code registries (Tool Catalog, Metric Catalog). Without a codified taxonomy, every new entity proposal repeats the same architectural conversation. The Phase 5a feedback_legacy_total_elimination.md memory captures the rule narratively but it isn't enforced anywhere — tribal knowledge that fails the moment a swarm or new engineer touches the data layer.

This ADR codifies the taxonomy, names it, sets the enforcement points, and establishes the planning-doc / ADR template convention so every new entity proposal has to explicitly declare which class it belongs to.

Decision

1. Three-layer mental model

The PropFlow data architecture is organized as three distinct layers:

The arrows always flow from catalog → events → spine, and from events → spine. Spine never references events; events don't import or call into catalog code at runtime (i.e., events-layer code does not call catalog modules; catalog-key strings stored in event rows like MetricSnapshot.metricKey are opaque identifiers, not runtime catalog references — that's a name reference, not a code-dependency).

A consequence: every "who triggered this?" question routes through the spine. The catalog tells you what exists; the events layer tells you what value/record was produced; the spine tells you who/what is involved. Joining requires walking from events → spine, never spine → events.

2. Two-class taxonomy: Canonical vs Derived

Within the events/values + spine layers, every entity is classified as exactly one of:

Canonical

Definition: Owns its own state. Cannot be reconstructed from other data. Deleting a canonical row + rerunning all producer jobs does NOT recreate it.

Test: "If I delete this row and rerun all producers, is it gone forever?" → Yes → canonical.

Sub-flavors of canonical (informational; same enforcement):

• Spine entity — identity + relationships (Person, Property, Lease, …)
• Role-relation — bridge between spine entities (TenantOccupancy, ProspectInquiry, VendorContact)
• Event log — append-only record of events (ActivityLogEntry, AgentTrace, AgentTraceStep, Conversation, Message, EmailIngestionRecord, SmsStatusRecord, BackfillAuditLog)
• Operational state — owned workflow state (RenewalSaga, Turnover, WorkOrder, Tour)
• Operational queue / log — operator workflow surface (BackfillReviewQueue, SyncLog, CalendarSyncFailure)
• Configuration — owned settings (Organization, AppSettings, PropertyLeasingSettings, PropertyMaintenanceSettings, CalendarIntegration, EmailIntegration)

Derived

Definition: Is a projection / cache / aggregate / rollup of canonical data. Can be rebuilt by querying or running a job over canonical data.

Test: "If I delete this row and rerun the producer job, do I get the same answer?" → Yes → derived.

Sub-flavors of derived (informational; reflected in naming suffix):

• Snapshot — time-windowed scalar aggregate (MetricSnapshot)
• Cache — short-TTL precomputed lookup (StatusCache)
• Stats / Summary — computed view-model for dashboards (DashboardStats, LeasingStats)
• Aggregate / Rollup — multi-dimensional summary (none today; reserved for future)
• Index — GSI-shaped denormalized lookup (none today as a standalone interface; reserved)

3. Naming convention (enforcement point #1)

Canonical entities: bare name. Person, Lease, WorkOrder, Conversation, AgentTrace, BackfillReviewQueue.

Derived entities: suffix that signals derivation. MetricSnapshot, StatusCache, DashboardStats, LeasingStats, future *Aggregate / *Rollup / *Index.

The naming-suffix list is fixed: Snapshot, Cache, Stats, Summary, Aggregate, Rollup, Index, View. New derived entities must use one. New canonical entities must not.

Rejected names (signal a class confusion at PR review time):

• ToolInvocation — sounds canonical; conflicts with the catalog's "events live inside Conversation+AgentTrace" rule
• MetricByPerson, CallByPerson — denormalized read-models masquerading as canonical
• *Mirror, *Copy — signals duplication of canonical state
• *Log (without further qualification) — ambiguous between event log (canonical) and derived rollup; prefer *EventLog or *Trace for canonical or *Stats/*Aggregate for derived. (Existing SyncLog and BackfillAuditLog predate this convention and are grandfathered — both are canonical append-only event-log shape; no rename required.)

4. Type-level annotation (enforcement point #2)

Every export interface in src/lib/data/types.ts carries exactly one of two JSDoc tags:

/**
 * @canonical Spine entity — owns identity attribute state.
 *   Spine trace: self
 *   See ADR-0018 §"Identity model" + ADR-0027 §"Canonical".
 */
export interface Person { … }

/**
 * @canonical Event log — append-only audit of every agent-tool call.
 *   Spine trace: Conversation.personId via conversationId
 *   See ADR-0027 §"Canonical · Event log".
 */
export interface AgentTrace { … }

/**
 * @derived from Conversation / WorkOrder / Tour rows in [windowStart, windowEnd].
 *   Rebuilt by: scripts/backfill-metric-snapshots.ts (per-property, cron-driven)
 *   Drift tolerance: ≤24h (dashboard polls every 3-5s — reads cached scalar)
 *   See ADR-0027 §"Derived · Snapshot".
 */
export interface MetricSnapshot { … }

Required slots:

• @canonical declarations require one line: Spine trace: <path> — where <path> is self, or the FK chain ending at Person (e.g., Conversation.personId via conversationId), or none (operational state that genuinely has no human, e.g., SyncLog)
• @derived declarations require three lines: from, Rebuilt by, Drift tolerance

Forbidden combinations:

• Both @canonical and @derived on the same interface
• Neither tag on any export interface in types.ts
• @canonical on a name ending in a derived-suffix (*Snapshot, *Cache, *Stats, …)
• @derived on a bare name not matching a derived-suffix

5. Drift guard (enforcement point #3)

A new vitest at src/__tests__/entity-classification.drift.test.ts enforces the rules above. The test:

1. Reads src/lib/data/types.ts source
2. Parses every export interface declaration + its preceding JSDoc block
3. Asserts each interface carries exactly one @canonical or @derived tag
4. Asserts @canonical declarations carry a Spine trace: line with self / a FK path / none
5. Asserts @derived declarations carry from:, Rebuilt by:, Drift tolerance: lines
6. Asserts name ↔ tag agreement (canonical names match bare convention; derived names match suffix convention)
7. Self-check: every fixture in the test carries a stated rationale (≥40 chars)

Failure mode: a new entity added without proper classification breaks CI. The error message points at this ADR and the convention.

This is the same shape as legacy-field-elimination.drift.test.ts (Phase 5a) and load-bearing-migration-infra.drift.test.ts (full-phase audit). Drift-guard-as-architecture-enforcement is the established pattern.

6. Diagram convention (enforcement point #4)

In architecture/entity-model.md Mermaid ER diagrams:

• Canonical entities render with bare labels: PERSON, WORK_ORDER
• Derived entities render with a marker prefix in their display label: METRIC_SNAPSHOT["📊 MetricSnapshot (derived)"]
• Catalogs render with a registry marker: TOOL_CATALOG["📖 Tool Catalog (code registry)"]
• Solid lines (||--o{) represent canonical FK relationships
• Dotted lines (..>) represent "derived from" projections — clearly distinct from FKs

Visual separation in the diagram makes the architectural claim visible at a glance. A new entity drawn on the canonical side that should be derived will look wrong immediately.

7. Atlas treatment (enforcement point #5)

The /admin/dev/atlas/ file-explorer surfaces the architecture as the operator sees it:

• Root tiles (entities that don't sit inside an Organization):
  • Organization — the tenancy orgs (org_jpco customer, org_sandbox test bench, org_propflow_staff internal). (The pre-2026-07-01 single shared org_default envelope was eliminated by the org_jpco tenancy split — PRs #2752/#2763.)
  • Vendor — cross-org per current schema (no Vendor.organizationId FK; implicit via property assignments)
  • Tool Catalog — global code registry; drill-in shows individual tools + their AgentTraceStep invocation count + recent invocations linked by Conversation
  • Metric Catalog — global code registry; drill-in shows individual metric definitions + current snapshot value + the underlying spine query that produces it
  • User (post-Phase-6a) — auth-side subject; Phase 6a backfill stamps User.personId linking to spine
• Inside Organization: Properties → Units / Leases / TenantOccupancies / ProspectInquiries; Persons → their relationships; the rest of the spine
• Drill paths for derived entities show their canonical source explicitly: opening MetricSnapshot shows "Derived from Conversation/WorkOrder/Tour" with a link to the producing query

8. ADR + planning-doc template (enforcement point #6)

Every new ADR proposing an entity must answer in its ## Decision section:

1. Class: canonical or derived?
2. If canonical: what's the spine trace path? (self / FK chain / none with operational rationale)
3. If derived: what does it derive from, who rebuilds it, what's the drift tolerance?

The ADR template at docs/adr/template.md is updated in this same PR to include these required fields.

Planning docs scoped at docs/data-model-migration/planning/ or its successor for Phase 6+ inherit the same convention: any new entity proposed in a planning doc must declare class before code lands.

Consequences

Positive

• Tribal knowledge becomes enforcement. Phase 5a's "single source of truth" rule is no longer a memory file someone might miss — it's a CI gate that fails red.
• Diagram + Atlas + types.ts agree. The same architectural claim is visible in three places (diagram annotation, JSDoc tag, Atlas drill-path) — any one of them can be the entry point for a new reader.
• New-entity discussions become 5-minute decisions, not 2-hour debates. "Is it canonical or derived?" is the first question; everything else follows.
• Phase 6 inherits the convention. When Phase 6a adds User.personId, the JSDoc @canonical tag with Spine trace: User.personId via Person.id makes the bridge explicit. Phase 6d's PersonRole lands cleanly as a role-relation per the ADR-0018 §3 pattern.
• Read-model PRs become safe. A future "we need a MetricLeaderboard for the dashboard top-10-callers widget" PR is no longer ambiguous — the entity is named with a derived suffix, tagged @derived, declares its rebuilder + drift tolerance, and drift-guarded. No risk of it accidentally becoming a competing source of truth.

Negative / Costs

• Every interface in types.ts (~70 today) needs a JSDoc annotation pass — one-time effort, scoped in Phase A.2 PR. ~210 lines of annotations to add. Mechanical, drift-guard catches misses.
• Adding a new entity gets slightly heavier — must include classification tag + name suffix + ADR/planning-doc class declaration. Roughly +5 minutes per new entity. The cost is small; the benefit compounds.
• Naming-suffix rule rejects some otherwise-natural names (e.g., ToolInvocation). Authors must pick a derived-suffix name (ToolInvocationCache, ToolInvocationStats) — or reconsider whether the entity is really derived. The friction is the point.

Neutral

• Existing entities that are mid-class (e.g., DashboardStats, LeasingStats) get clarified. Both are derived; their names already match the suffix convention; only the JSDoc tag is new.
• Phase 5a's drift guards stay independent. This ADR's drift guard is additive — legacy-field-elimination.drift.test.ts (Phase 5a) and load-bearing-migration-infra.drift.test.ts (audit) keep doing their jobs. The new entity-classification.drift.test.ts covers a different invariant (taxonomy adherence).

Alternatives considered

Alternative A — No taxonomy; keep "single source of truth" as memory-file convention

Rejected because: the Phase 5a audit + this conversation already surfaced the failure mode. MetricSnapshot.byPersonId and ToolInvocation were both proposed as "obvious" next-step entities; without an enforced taxonomy, they'd land via PR review judgment calls rather than CI gates. Tribal knowledge erodes over years; the migration the next team inherits would look like the one Phase 5a just spent two weeks undoing.

Alternative B — Many-class taxonomy (separate classes for spine / role-relation / event-log / operational-state / snapshot / cache / aggregate / …)

Rejected because: sub-flavors are informational, not enforcement-worthy. The architectural test that matters is "can this be reconstructed from other data?" — a binary. Splitting into 7-8 classes adds vocabulary without changing the enforcement rule; reviewers and CI need only the binary call. Sub-flavors live as docstring conventions and naming suffixes — not as separate tags.

Alternative C — Event-sourcing as the canonical pattern

Rejected because: PropFlow's existing data shape (DDB single-table, role-relation entities, lazy aggregation) is not event-sourced. Retrofitting CQRS / event-sourcing post-Phase-5a would be a full architectural rewrite. The taxonomy in this ADR is compatible with our existing pattern (some entities are append-only event logs; others are state-bearing); it doesn't require event sourcing.

Alternative D — Defer the taxonomy until Phase 6 closes

Rejected because: the Phase 6 work (User-domain bridge via Person) is exactly the kind of work that NEEDS the taxonomy to avoid drift. Without ADR-0027, Phase 6 has the same risk Phase 5a inherited from the legacy model — accumulated denormalization patterns that take a future migration to undo. Lock the convention before the work starts, not after.

Phase A adoption (this ADR's rollout)

The adoption work that operationalizes this ADR splits across three sequential PRs:

Each PR ships independently with /review-turns, merges on clean verdict, then the next PR rebases on main and proceeds.

References

• ADR-0018 — Person identity model — the spine
• ADR-0019 — Organization model — org scoping
• ADR-0020 — Person as universal human spine — Phase 6 extension
• Phase 5a final report — the closure event that made this taxonomy explicit
• Full-phase audit report — the audit that surfaced the catalog/spine architectural question
• legacy-field-elimination.drift.test.ts — the precedent for CI-enforced architectural invariants
• metrics-platform.md — the Metric Catalog's content reference (this ADR codifies the shape, not the contents)