0068 — The medic revives and pokes on the session pulse: one nudger, reboot-aware, and nothing decided by a name
- Status: Accepted 2026-10-18. Recorded at decision time (ADR-0004). Built in
PropFlow-Technologies/local-bin#627 (fleet-medic, nudge, watcher-doctor, start-driver); this repo carries the record and the catalog entries. It supersedes one row of ADR-0030 decision 4 and the check-in clause of ADR-0067 What keeps working; see What this supersedes. - Date: 2026-10-18
- Deciders: Gera (they/them), who approved this design on 2026-10-18 after the 8-day audit of the Driver helpers, and ruled out the obvious shortcut — a list of the sessions worth reviving: "what if we have it in the future, like for different sessions". The rule has to hold for any session, by what it is doing, never by what it is called. The same day: "I don't want things keeping sessions alive when I don't want that anymore."
- Systems:
- fleet-medic (local-bin, launchd
co.propflow.fleet-medic, 300 s): relaunches dead hosts and pokes idle sessions. It now decides both from the pulse. - the session pulse (
relay/src/pulse/, ADR-0006): the one classifier. It gains a reader that ACTS on its states. - nudge and watcher-doctor (local-bin): the Driver check-in they armed is retired, and so are nudge's 60 s floors.
- ccswitch walled-heal: becomes the only thing that moves a session to another account.
- fleet-medic (local-bin, launchd
What was measured
An audit of the mini, 2026-10-11 to 2026-10-18, followed every helper prompt in 371 transcripts to the next outside prompt and recorded what the session did next.
| helper | count | led to real work (commit or edit) | the waste |
|---|---|---|---|
| fleet-medic relaunch | 113, all on Oct 17–18 | 5 % | 75 in two hours after one reboot, load 45; 13 of them into sessions a human had PAUSED |
fleet-medic seat move (ccswitch move) |
312 moves | 4 % | 972 refusals, 993 alarm lines, contending with ccswitch for the same seats |
| fleet-medic idle goal-count nudge | 658 | 15 % | 17 % landed on a session that had already said it was finished or waiting |
Driver check-in (nudge --_watch … "DRIVER CHECK-IN", armed by watcher-doctor every 180 s) |
571 | 7 % | 33 % into a PAUSED session; 57 % fired on a 60 s floor, not the 600 s idle rule |
One 60 s watcher put 881 Compact landed; re-read your keep-list and continue. prompts into a single session in three days, 1.59 B input tokens, until a human paused it. After the 02:43 reboot on 2026-10-18 the medic's first tick read all 83 leases as outages ("host died without a clean exit") and relaunched 64 of them over two hours. By contrast ccswitch's walled-heal, which only acts on a session that was working when it hit the wall, resumed 98 % of the 136 sessions it moved.
All four used crude rules: "I saw this host alive once", "this sid8 is on a list", "the screen has not changed", "there is an open block, poll every 60 s". None of them asked what the session was doing. The fleet already had a component that answers exactly that: the pulse classifier writes one verdict per session — working, paused, waiting, needs-kickoff, done, dead, or null — to ~/.claude/jobs/pulse/snapshot.json every 120 s.
The decision
- A dead host is relaunched only if it had unfinished work when it died. The medic records, every tick, the pulse state it reads for each session whose host is alive. That record lives in
~/.claude/jobs/medic/state.jsonbecause the snapshot only says "now", and after a death or a reboot "now" isdead. A relaunch needs all of these:- The remembered state is
workingorneeds-kickoff(revive_states).paused,waiting,doneand null are refused, and so is a session with aloop/<sid8>/PAUSEDmarker. - The record does not predate the lease's last renewal by more than an hour. (The state is recorded once per tick, and under load ticks landed 30–40 minutes apart, so a tighter bound would refuse a session that was working right up to its death.)
- The current snapshot is fresh and does not show the sid running.
- Everything the medic already proved before relaunching still holds: no deliberate close, no live pane, no process resuming the sid, the restart caps.
- A lease never classified while alive is history, and is left alone silently.
- The remembered state is
- A reboot is not an outage. A lease last renewed before
kern.boottimelost its host to the reboot, along with every other session on the box. Those are relaunchedworkingfirst, at most 2 per tick, and only while the 1-minute load average is under the CPU count. A boot time that cannot be read takes the same stricter path. - Unknown is not permission. If the snapshot is missing, unreadable, of another version, older than 900 s, stamped in the future, or marked
blindby the collector, the medic revives nothing and pokes nothing. Every tick says so (medic PULSE … — reviving nothing, poking nothing, tick linepulse=UNKNOWN). Wedge-kill needs no pulse and is unchanged. - One nudger. The medic's idle nudge and the Driver check-in become one decision, in the medic.
- It pokes a session only while the pulse reads it
needs-kickoff(poke_states): open work, and nothing armed to wake it. - At most one poke per session per 10 minutes.
- Each poke is graded at the next decision. A tool use after it that is not a pure read counts as work. A reply, file reads, or an unreadable transcript is a strike.
- Two strikes in a row stop the pokes until a human types in the session. A human prompt is one whose text starts with none of the stamps every injector puts on its own text (
[fleet-medic],[msg from … · sid8],[webhook],<task-notification>,ccswitch:…). Claude Code'sorigin.kindreadshumanfor all of them, measured on three transcripts that day. - PAUSED is always honoured.
- The
GOAL: n of m/WAITING ONparks stay, re-armed by the same human prompt.
- It pokes a session only while the pulse reads it
- The Driver check-in is retired.
watcher-doctor --repairarms nothing. Every 180 s it retires each live check-in throughnudge --retire-checkin --sid <sid>, a paused Driver's too. That verb stops only that session's check-in watcher(s); the session's own one-shot and--everywakes survive. A watcher keeps the script it was armed with, so the existing ones can only be ended from outside.- A Driver with no watcher reads
UNWATCHED, the designed state, notDARK. It is never repaired and never alarmed. start-driverarms no wake.nudgerefuses any--idle/--everyunder 600 s, and an open block shortens a watcher to 600 s, not 60.
- The seat move is retired. ccswitch's walled-heal owns account moves. One owner per job is the 2026-09-11 incident's lesson, and the medic's own docstring already warned against two.
- No session names anywhere. The medic's built-in watch list (five sid8s → names) is deleted, and a
watchkey left inconfig.jsonis reported as no longer read. Every rule above is a function of state.
How the systems relate: by files, never calls (ADR-0003). The relay writes pulse/snapshot.json. The medic reads it, and writes its memory to its own state.json. watcher-doctor reads the process table and calls nudge, a sibling tool on the same rung. Nothing calls the relay. The medic pins PULSE_SNAPSHOT_VERSION 1: a version bump, which the collector makes "when a field's MEANING changes", reads as unknown and stops the medic acting until it is updated. That is the safe direction.
What it does to today's sessions — measured, not inferred
fleet-medic explain (read-only, new) run with the built tool against the live snapshot on the mini, 2026-10-18 22:16 CDT (load 29, snapshot 15 s old, 257 rows):
| live sessions (61) | poke verdict |
|---|---|
| 7 | eligible: the pulse reads them needs-kickoff. All seven are non-Driver sessions holding 1–3 open blocks with nothing armed to wake them. Three are Gera's own tabs (6f9293a8, b8db9d59, c05acd12). Each is poked only after its screen has been still for 15 minutes, at most once per 10 minutes, and at most twice without work — then never until Gera types in it. |
| 20 | not poked: waiting |
| 24 | not poked: the pulse cannot say (null) |
| 10 | not poked: a human PAUSED them |
No dead lease was revive-eligible, because the medic has no memory yet. The first tick after deploy starts recording. Until a host has been seen alive under the pulse, its death relaunches nothing.
⚠️ The consequence that needs a person, stated plainly. In this classifier a live Driver with an open task is its own wake path (collector.ts driverBoundFor: "a Driver row answers true about itself"). So it always reads waiting, never needs-kickoff. Read literally, as approved, this rule therefore never pokes a live Driver and never relaunches a dead one that was waiting. The Driver check-in it replaces is gone, so an idle Driver now wakes only on its own events:
- a sub-agent finishing,
- a decision answer,
- a webhook,
msg,- Smith's Temporal watcher for the lanes armed there,
- a human.
That matches ADR-0067 ("a Driver that dies stays dead"). It also matches what the audit measured for the check-in: 7 % led to work. But it is a choice, not a fact. The other reading is "unfinished work and nothing coming to wake it". It would need the classifier to stop counting a Driver with no open blocks as its own wake path: a classify.ts change and a PULSE_SNAPSHOT_VERSION bump. It is not taken here. revive_states and poke_states are config keys, so the medic needs no code change if that changes.
What this supersedes — precisely
| Record | Superseded | What stands |
|---|---|---|
| ADR-0030 decision 4, row "Waking the Driver that owns it" | The nudge check-in, armed for every Driver. Retired. Idle pokes are fleet-medic's, on the pulse. |
The rest of the table, and its principles. |
| ADR-0067 What keeps working, bullet "The idle-Driver check-in" | "Still the Driver waker". It is not. | Everything else in ADR-0067. |
| fleet-medic remedy 3, the seat move (code, no ADR) | Deleted. | ccswitch walled-heal, unchanged. |
fleet-medic's watch list and resurrect: seen/watch, nudge_idle: watch/all (code) |
Deleted. The old values read as "on", pulse-gated. | resurrect: off and nudge_idle: off still switch each half off. |
Per ADR-0023 decision 6, ADR bodies are not rewritten. ADR-0030's and ADR-0067's Status lines carry the pointer here, appending no line.
Inventory of what implemented the old way (ADR-0004 part 2), and what happened to each (part 3, in the same local-bin PR):
| Surface | Fate |
|---|---|
watcher-doctor --repair arming nudge --idle 600 "$(watcher-doctor --checkin-text …)" |
removed; --repair retires live check-ins instead |
start-driver arming the same at spawn |
removed |
| the ~19 live check-in watchers on the mini (old code in memory) | retired by the first --repair sweep after deploy |
nudge MIN_INTERVAL=60, the 60 s open-block floor |
600 s, both |
nudge --stop's warning that the doctor would re-arm |
removed (it no longer would) |
fleet-medic spent_seat + its ccswitch import, the limit-menu Esc + "continue" rescue |
removed |
tests/control-36-dark-watcher.sh (pinned DARK → re-arm) |
deleted; its nudge-only half kept as tests/nudge-nap-lets-a-trap-run.sh |
tests/control-66-every-driver-gets-a-check-in.sh |
now control-66-no-driver-gets-a-check-in.sh |
fleet-bake's "zero watchers" and "dark lanes > 3" regressions |
replaced by "a check-in survived retirement" |
| the Driver manual's "your check-in is armed for you" and re-arm instructions | rewritten |
Deploy, and how to undo it
- Deploy: merge local-bin #627.
doctrine-pull(LaunchDaemon, every 300 s) fast-forwards~/.local/bin, and every tool reads its new file on its next run. Nothing restarts.- The first watcher-doctor sweep (180 s) retires the live check-ins.
- The first medic tick logs the retired
config.jsonkeys once and starts recording pulse states. - Any live
nudge --everychain under 600 s ends at its next fire.
- This repo's half (this ADR, the catalog entries) reaches the Systems tab when the mini's agentflow checkout (
live) moves past it. Until then, watcher-doctor's oldevery live Driver armedsignal reads stale against the new footer. That is noise, not a fault. - Switches without a revert, all in
~/.claude/jobs/medic/config.json, read every tick:"nudge_idle": "off"or"resurrect": "off";- widen
poke_states/revive_states; touch ~/.claude/jobs/medic/STOPstops the medic outright.
- Full revert: revert the local-bin PR. That restores the check-in arm, the seat move and the watch list together. Then supersede this ADR.
Options not taken
- Keep a list of sessions worth reviving (the medic's watch list, or "only Drivers"). Gera ruled it out: it fails the next session nobody listed, and a name says nothing about whether the work is finished.
- Fix the 60 s floor and keep the check-in. That leaves two pokers for one job, still blind to state. 33 % of check-ins landed on paused sessions even with a pause check in the code, because live watchers run the script they were armed with.
- Revive everything after a reboot, just slower. Fewer per tick still means everything, and that is the reboot storm spread over more time.
- Let the medic re-derive the state itself from screen, blocks and tasks. That is the sixth classifier ADR-0006 exists to prevent.