2026-07-24 → 07-25 · harness-measured on propflowai.co (smoke user, real prod data) · 24-agent adversarially-verified code audit · 6 fix iterations, 10 PRs shipped
The app's slowness is real, growing, and now mapped. Page HTML arrives fast everywhere; the 11–35s a PM actually waits is client-side API waterfall on top of a database access layer that re-reads whole tables per request. The "database growing" suspicion is confirmed — but the growth hurts through specific, fixable patterns, the worst being full-table Scans whose cost is O(everything ever written): the table is dominated by agent traces and messages, so every Clara call made the tenants page slower. Six of the worst offenders were fixed overnight (all merged, deployed, measured). The remaining ~11s all-page floor is structural — layout-level portfolio aggregates, per-instance caches that don't survive serverless fan-out, and a 600–740KB client bundle — and the ranked roadmap below is the path to a genuinely fast app (~1–2s pages).
/api/tenants/[id] (staff read — was a 457k-item table Scan)/api/tenants/[id] 404 miss (pure scan cost)/api/tenants/[id]/summary repeat open (LLM + full waterfall each time)/api/dashboard/stats warm repeat (single-slot cache evicted itself)/api/vendors (824 vendors × 3 reads each → batched)| # | Fix | PRs | Measured result |
|---|---|---|---|
| 1 | Full-table Scan on staff tenant lookup killed. findOccupancyByIdAnyOrg scanned all 457,246 items to find 1 of 202 occupancy rows (waste ratio 2,263:1) — replaced with a bounded 4-org parallel point-query fan-out, mirroring the prospect twin fix already in the codebase. Also fixed for free: work-order create, voice inbound, PMS sync paths that passed empty org. | #4500 | API 11–13s → 2.2s; miss 22.4s → 0.79s; tenant page settle 34.9s → ~19s (page pays the read 3×) |
| 2 | /api/dashboard/stats cache actually caches now. The old cache held exactly ONE entry — any two concurrent keys evicted each other (~100% miss under real traffic) on the endpoint fetched by 53 pages every 30s. Now: keyed TTL map, 50-entry LRU, in-flight coalescing. | #4502, #4513 | distinct keys warm-hit 0.24–0.60s vs 2.8–3.6s cold (~10×), all keys held simultaneously |
| 3 | Tenant AI summary no longer regenerated per open — cached by inputs-hash. | #4511, #4517 | partial: LLM call eliminated (12.7→6.7–9.0s) but the route still ran its data waterfall to compute the hash → fix 5 |
| 4 | Polling hygiene fleet-wide: usePolling in-flight guard (ticks no longer stack when a response outlives the interval) + deps-change re-fire preserved; prospects page raw 5s setInterval converted. | #4518, #4523 | verified by revert-then-pass regression tests; stops request pile-ups on every polled surface |
| 5 | Summary endpoint whole-response cache (90s TTL, org-scoped key, auth before cache) + assembly parallelized (7 sequential awaits → 3 waves). | #4521, #4526 | repeat opens 9.0s → 0.29s (~25×), byte-identical payload |
| 6 | Vendors roster contact reads batched: 3 DDB round-trips × 814 vendors (~2,450 reads) → ~825, exact payload parity, op count now independent of vendor count. | #4537 | reads −3× but median wall-clock unmoved (min 5.8→3.2s) — remaining cost is 812 claim queries that yield only 17 phones + 15 emails. Product decision below. |
Full-app harness re-sweeps confirm the targeted wins (tenant detail −16s, renewals board 13.7→5.7s) and show the ~11s floor on all other authed pages is unchanged — expected: the floor is structural, not endpoint-local. Sweeps ran during heavy overnight automation traffic and fresh deploys, so page-level medians carry noise; endpoint-level and isolated-curl numbers above are the clean signal.
Every authenticated page pays the same tax before its own data loads:
| Layer | Cost | Mechanism |
|---|---|---|
| HTML/TTFB | 35–100ms | fine everywhere — this is why "the app is slow" never showed in uptime checks |
| JS bundle + hydration | ~1–5s | 600–740KB compressed per page, essentially no code splitting; Sentry Replay statically imported; dashboard is a pure client component (zero server data, full mount→fetch waterfall) |
| Layout-level fetch fan-out | ~3–9s | /api/dashboard/stats (53 pages, med 3.1s), /api/leasing/stats (57 pages), /api/properties (68 pages), auth/me ×2, settings ×2 — fired on every page, re-polled every 5–30s |
| Page-specific reads | 0–22s | the scans / N+1s / unpaginated reads in the findings list |
| CDN cache | 0% hit | every API GET is x-vercel-cache: MISS; in-process caches die with each of Vercel's many short-lived instances |
Suspicion confirmed, with a precise shape: the table (456k items / ~976MB) is dominated by AgentTrace (66,515 rows), messages and activity logs — not by tenants (202 occupancy rows) or vendors. Any code path that Scans the table or reads a whole entity set pays for all of it. So the app got slower as Clara worked, not as the business grew.
computeDashboardStats reads six whole entity sets unscoped per cache miss; /api/conversations reads 2,500 conversation metas + all tenants for a 20-row page; email-ingestion partition (8,193 rows, ~290MB scanned) pulled on 4 routes' mounts; /api/turnovers even runs a write-capable auto-create pass per GET.Overnight work removed the worst tails. Getting from ~11s pages to ~1–2s pages is these six items, in order of leverage:
| # | Change | Expected effect | Effort / risk |
|---|---|---|---|
| 1 | Cross-instance caching for the read-mostly hot endpoints (dashboard/stats, leasing/stats, properties, settings): CDN s-maxage + stale-while-revalidate on scope-safe GETs, or a shared store (Vercel KV/Redis). In-process caches barely help — Vercel fans requests across instances, so most "warm" requests land cold. | the every-page tax ~3–9s → <300ms; biggest single lever | medium; needs per-endpoint scope-key review (auth-scoped cache keys, same pattern as #4521) |
| 2 | Materialize dashboard aggregates: write-through counters or a periodic snapshot row instead of computeDashboardStats re-reading six entity sets per miss. The Metrics Platform snapshot machinery already exists. | stats compute O(portfolio) → O(1); unblocks #1's short TTLs | medium-high; one aggregate at a time |
| 3 | Fetch layout data once, share via provider: stop each page/component refetching dashboard/stats/leasing/stats/properties independently (dedupe auth/me ×2, settings ×2); lengthen poll intervals on background surfaces. | removes 5–10 requests per navigation | low-medium, pure frontend |
| 4 | Pagination + lazy detail on heavy lists: conversations (2,500 metas → 20 rows), vendors (305KB, 824 rows; contacts column renders 17 phones out of 814 rows), work-orders full-view N+1, email-ingestion. | list pages stop scaling with history | medium; some need product sign-off (below) |
| 5 | Bundle diet + server-render the shell: code-split heavy libs, optimizePackageImports for phosphor, dynamic-import Sentry Replay, server-render list first-pages (they already SSR then re-fetch identical data client-side). | 1–5s hydration head shrinks; pages paint with data | medium, incremental |
| 6 | Data hygiene + remaining scans: TTL/archive agent traces out of the hot table (or move to their own table); fix PMS-credential connect scan; turnovers auto-create out of GET; middleware auth round-trip (edge PoP → us-east-1 ×3 reads per request, plus an internal HTTPS hop). | kills the remaining O(history) costs + ~2–3s middleware tail | scan fixes low; trace TTL needs a data-retention decision |
OCCID#<id>): makes tenant-by-id O(1) for any caller and deletes the org fan-out — schema change + backfill, so it's flagged rather than shipped. The shipped fan-out is already fast (0.3–2s); this is optional polish.docs(...)-titled PR touching a .ts file fails it.[id] routes resolved to real prod entities), records TTFB / load / network-settle / every /api/* call's start-offset + duration + cache status. Read-only GET navigation only. Runs: baseline2 (pre-fix), after-fixes, after-fixes-warm, final (config-identical to baseline).aws dynamodb COUNT queries). 59 findings survived.