Architecture plan · PropFlowAI
A self-hosted, operator-facing view of every route in the product — which pages get used, by whom, from where, how fast, and where they break. Plus one production landmine found along the way that has nothing to do with the feature.
Blocking finding · unrelated to this feature
describe-time-to-live --table-name propflow-prod returns
DISABLED. Meanwhile 19+ modules write a ttl
attribute to that table expecting auto-expiry. None has ever fired.
Confirmed writers include email delivery logs, tour outbox, status cache, dispatch log, MFA and login locks, cron heartbeats, native auth codes, NTV escalation markers, renewal sagas, the Better Auth adapter, and the investor deck's own flood-guard counters.
Two consequences. First, every one of those subsystems has unbounded row
growth today — nobody is paged about it because DynamoDB scales silently;
you just pay for it. Second, turning TTL on is table-wide and one-way.
Within roughly 48 hours DynamoDB would begin deleting expired rows across all 19
subsystems simultaneously, including auth sessions and renewal sagas, using
ttl values written by code that has since changed meaning and has never
been re-validated.
The first draft of this plan called enabling TTL "a one-line side-effect." That was the most dangerous sentence in the document. It needs its own ticket and its own audit. This plan therefore never touches production's TTL setting — the feature uses a separate table instead, which gets TTL at creation with zero blast radius.
Every row below was checked against the repo or AWS, not assumed
| Fact | Where |
|---|---|
| Middleware runs on every non-static request, Node runtime | src/middleware.ts |
| The request hook is error-only — not a universal request wrapper | src/instrumentation.ts |
| 82% of API routes already funnel through one wrapper | lib/platform/api/route-helpers.ts |
| Vercel Analytics is already mounted; Sentry samples 25% of traces | app/layout.tsx · instrumentation-client.ts |
| Canonical right-side slide-over primitive already exists | components/primitives/FullScreenDrawer.tsx |
| Correct beacon transport already solved: keepalive fetch normally, sendBeacon on unload | app/(standalone)/deck/DeckViewer.tsx |
Polling volume, measured
There are roughly 28 live usePolling() call sites, most on a 5-second
interval — dashboard stats, leasing stats, the work-order table, approval queue,
properties, turnovers, prospect detail, mass-sends, billing.
One property manager with the dashboard open generates about 1,440 requests per hour, or 17,000 over an eight-hour day. Ten concurrent users is roughly 170,000 requests per day — every one of them an API call, not a page view.
So the design splits by row family, not by an exclude list. Page navigations get raw event rows plus rollups. API routes get rollups only, never raw rows — counters, latency histograms, and status classes. A hand-maintained exclude list rots: every future poller becomes another decision nobody remembers to make.
Traffic class is a derived property of the request — human navigation, background polling, webhook, cron, internal, bot — so nothing is dropped. Polling and bot traffic stay queryable, just segregated so they can't distort the default view. The upside: the Performance tab honestly covers all 356 API routes on day one, which an exclude list would have made impossible.
Four defects in the v1 data plane, found by an independent Codex review
v1 wrote ADD uniqueSessions. There is no such thing — a numeric
ADD cannot know whether a session already counted, and a DynamoDB string
set grows the item toward the 400 KB cap while parking identifiers in a hot row.
Replaced with a conditional dedup marker that gates the counter: a
PutItem with attribute_not_exists succeeds exactly once per
session, route, and day, and only then does the counter increment.
v1 sharded the base table 16 ways, then projected every event into a single
GSI1PK = TRAFFIC_ROUTE#<route> — rebuilding, inside the index,
exactly the hot logical partition the sharding existed to prevent. The index key now
carries date and a small shard, and the drawer fans out across the range.
Server duration, document TTFB, soft-navigation time, and LCP are different physical measurements and can't share a histogram. Worse, five buckets topping out at three seconds can't yield a percentile — if 90% of traffic lands in the 500–1000 ms bucket, p50 could be 510 ms or 990 ms. Now: one logarithmic histogram per metric.
Middleware sees a pathname, not a route template — it doesn't know
/tenants/t_123 matched /tenants/[id] — and it can't observe
the eventual status or duration. Meanwhile LCP, INP, and dwell aren't knowable at
navigation commit. Now: server facts come from the server, browser facts from the
browser, joined on a navigation ID, as four idempotent event types.
Codex claimed no mechanism exists to wrap 356 API handlers. There is one, and it's
already there: withErrorHandling is used by 293 of 356
route files. Instrumenting a single function yields 82% API coverage with correct route
templates and real server durations, and a drift test closes the remaining 63 over time.
It also recommended jumping straight to ClickHouse. That's over-built for this volume — so rather than argue, the plan pre-commits the trigger to switch: sustained traffic above 5M page events per month, a third drawer question needing a bespoke rollup, or the first ad-hoc filter the rollups can't answer.
Two sources, four idempotent event types
| Event | Source | Fires | Carries |
|---|---|---|---|
| page_view | browser | navigation commit | navId, route, referrer, session, viewport |
| page_vitals | browser | as they settle | TTFB, FCP, LCP, CLS, INP |
| page_exit | browser | pagehide / hide | dwell |
| api_request | server | handler completion | route template, method, status, server ms, org, user |
Storage lives in a dedicated propflow-traffic table, not
the primary single table. Single-table design is a modeling technique, not a mandate to
co-locate workloads: a telemetry firehose in the main table means analytics write
pressure competes with customer domain traffic on shared indexes, inflates backups and
restore time, and lets a traffic spike touch core operations. It also, conveniently,
is how the feature gets TTL without going near the finding at the top of this page.
ip, geo, organizationId, and userId
are derived server-side; the client payload is untrusted and
field-allowlisted. Honest write cost is 7–9 write units per page event
once index projection and every rollup dimension are counted — not the "two writes" v1
claimed.
Two places where a plausible design produces confident wrong numbers
Silent failure is not acceptable here. Best-effort logging is right for a low-volume investor page, but on a traffic dashboard it means you cannot distinguish "zero traffic" from "telemetry is broken." Every rollup carries a schema version, sampling rate, and last-write timestamp, and the page renders a freshness and health banner. If ingestion is down, the page says so instead of rendering a confident zero.
Aggregates never come from a paginated read. The drawer promises filters, group-bys, and percentiles — OLAP questions. Folding those in a serverless function over a paginated index read is only correct at low volume, because the first 1 MB page is not a representative sample. So every drawer aggregate is served from a precomputed rollup; the raw event list is the only paginated element and is labelled "most recent N," never summed.
Sampling stays off the error path. v1 proposed sampling APIs at 10%, which makes error rates lie — rare failures vanish entirely. Sampling, if ever enabled, applies only to successful high-volume polling rollups, and every sampled row carries its probability so queries can weight correctly.
Requires an explicit decision, not a default
v1 contained a straight contradiction: it recommended HMAC-hashed IPs and promised an exact-IP drawer. Those are mutually exclusive, and a rotating salt additionally breaks all cross-day visitor analysis.
Storing raw IP alongside user ID, organization, timestamp, and route viewed is employee and customer activity monitoring, not merely telemetry. Being on an admin-only page doesn't remove the obligations: a retention line in the privacy policy and trust center, genuinely restricted and audited access, a deletion path for an erasure request, and an internal decision about whether staff page-view history is something worth retaining at all.
Non-negotiable regardless of that choice:
tab, page,
sort, filter, and utm_* are allowlisted.
Tokens, emails, and redirect targets never persist.
data-track — never blanket autocapture,
which would record element text and land tenant names, unit numbers, and vendor
phone numbers in the analytics store.
One concern per pull request
| PR | Scope |
|---|---|
| 0 | Separate ticket, not this feature — audit and enable TTL on the production table |
| 1 | New propflow-traffic table with TTL at creation, writer library, event taxonomy, tests |
| 2 | Ingest route — server-derived fields, schema allowlist, rate limit, background write, health counters |
| 3 | Instrument withErrorHandling → 293 API routes, plus a drift test for the other 63 |
| 4 | Client beacon — four event types, keepalive/beacon transport, bfcache handling, idempotent nav ID |
| 5 | Route manifest generator, surface rule list, client route matcher |
| 6 | The page — Pages tab, drawer, dev-tools registry entry |
| 7 | Visitors, Geography, Performance, Live tabs |
| 8 | data-track clicks on high-signal controls |
The existing /admin/dev/deck-traffic page stays as-is. Its semantics are
email-identity based — investor unlock, then session, then slide dwell — not route based.
Folding it in would force a lowest-common-denominator schema on both. Cross-link instead.
Defaults apply if nothing is said
Confirm raw IP with 90-day expiry and the policy and erasure obligations that come with it, or trade the exact-IP drawer away for hashing. You can't have both.
Default → raw IP, 90 days, write the policy lineThe full eight-PR plan, or a narrowed first cut — pages, route rollups, and the drawer only — to validate the shape before committing to the ingestion surface.
Default → full plan, phased as aboveOpen it as its own investigation now? It's a live production issue independent of this feature, and it's the highest-value thing this session found.
Default → file as a finding, don't act