2026-08-21 · Fede's question: "are we using Temporal to track where things are in the human-in-the-loop escalation?" · POC ran end to end at The Willows bench
The one-paragraph answer. No — escalations are the only human-waiting lane NOT on Temporal. Their state is database rows swept by a once-a-day cron. Renewals, the older PM-reminder lane, and maintenance comms all already run durable Temporal workflows, and the company rejected the cron-sweep shape in writing when it built them. The prototype below puts one durable clock on each escalation and proves, in a 75-second live run, everything the cron can't do: deadlines that fire on time, chasing that survives a crash, a stop that's instant when staff reply, and stale holds that expire instead of muting someone forever. Recommendation: adopt — as Decision 6, option A in the escalation architecture doc (a fifth kind on the existing reminder workflow, not new machinery).
| Lane | State lives in | The clock |
|---|---|---|
| Renewals | Temporal workflow (+ PMS mirror) | Durable per-case workflow — waits on real events, own timers (ADR-0025) |
| PM action reminders (applications, countersigns, forwarded questions) | Temporal workflow | Durable per-action cadence, live per-property config, "remind now", resolves on signal (ADR-0104) |
| Maintenance comms | Temporal workflow | Long-lived, terminal only at close/escalate (ADR-0053) |
| Escalation matters (the new single-channel lane — Camellia & Willows) | Database rows | Once-a-day cron sweep, 9am Mountain |
The history explains the oddity. Temporal was adopted (mid-May) precisely because the previous database-rows-plus-cron design let nine renewal cases silently stall for days. When the PM-reminder lane was built, a cron sweep was considered and rejected: "reinvents state Temporal already keeps durably." The team's own staging rule (in the review-inbox planning doc) says it plainly: "the moment we want auto-expiry, reminders, or reliable retrying … the timer belongs in the workflow — we do NOT add a cron/sweeper to fake the timer." The matter lane, built fast during the escalation redesign, is an accidental exception to a rule the company already holds — no document argues for keeping it on cron.
Live cost, from the day this was written: a gas-leak escalation's 4-hour response deadline expired at 3:24am and nothing noticed until the 9am sweep — five and a half hours late. A deadline expiring just after 9am waits until tomorrow.
Side note from the competitor research: none of EliseAI / Colleen / Funnel / Zuma publish how they hold follow-up state internally — the corpus only documents their user-facing task queues. No industry answer to copy; this is ours to decide.
One workflow per escalation (matterChaseWorkflow), run against real Willows bench matters opened through the production escalation code, with real reminder emails through the production email builders. Fully isolated: local Temporal server, dedicated queue, a worker that refuses to start if pointed at the cloud. Two matters, compressed clocks (15-second deadline, reminders every 15s, expiry after two unanswered reminders):
t+6s two real matters opened; workflows started; worker up
t+13s worker KILLED (hard, mid-wait — nothing has fired yet)
t+20s the deadline passes WITH NO WORKER RUNNING
t+26s worker restarted → both owed deadline-breach stamps fire ✓
t+35s staff reply on matter A → chase stops INSTANTLY, zero further reminders ✓
t+35-74s matter B never answered → 2 reminder emails (real, delivered) ✓
→ reminder budget exhausted → AUTO-EXPIRED, hold released ✓
ALL CHECKS GREEN
The kill-and-restart is the heart of it: the clock state lived on the Temporal server, not in the process, so a crash, deploy, or restart loses nothing — every owed step fired the moment a worker returned. Under the cron design, process death costs nothing only because the clock only ticks once a day; you've traded fragility for blindness.
We gave each open question its own stopwatch that keeps running even if our servers restart. We shot the timekeeper mid-countdown, brought a new one in, and every alarm still went off — then a staff answer silenced the right stopwatch instantly, and an ignored one gave up politely instead of holding the resident hostage.
| Today: daily cron sweep | POC: one workflow per matter | |
|---|---|---|
| Deadline accuracy | Up to ~24h late (next 9am) | Fires at the deadline |
| Stop when staff reply | Next sweep re-reads state (a nag can't double-send, but nothing reacts) | Instant, via signal + fail-closed re-check |
| Reminder cadence | One global daily pass for every matter | Per-matter interval & budget; per-property config; "remind now" possible |
| Auto-expiry (Decision 2) | Needs a second sweep to build | One more timer in the same loop — demonstrated |
| Crash / deploy behavior | Nothing lost — because nothing is happening between sweeps | Owed steps fire on worker return — demonstrated by hard kill |
| "Where is this matter?" | Infer from row fields | Query the workflow: phase, nags sent, time waiting |
| Ops surface | One Vercel cron; trivially inspectable | Rides the EXISTING workers & queue — but see costs below |
Adopt, the small way. Not a new workflow: add an escalation_matter kind to the existing, battle-tested per-action reminder workflow (it already has cadence config, budgets, remind-now, signal resolution, and workers deployed). Wire two seams: opening a matter starts the cadence; the staff-reply release sends the resolve signal. Keep the daily cron as a backstop for one observed week, then retire it. This is Decision 6, option A in the escalation architecture doc — nothing ships until Fede picks there, and it arms per property, Willows first.
POC code + green run transcript: PR #6077 (scripts/temporal-matter-poc/, bench-only, cannot touch prod queues).