ccswitch · Round 12 · 2026-09-06/07 · session ccswitch
Outcome first: the switch moved nobody because Claude Code’s own usage-limit wait (“continuing automatically at 10pm”) suppressed the Stop the respawn needed. The fix ended the turn at the wall (autoContinueAtUsageLimit: false), made the 80% switch interrupt instead of wait, healed the wall from the transcript, moved the meter from 15 minutes to 60 seconds, and made the whole path provable on demand (fleet-e2e --switch, four consecutive passes). PITFALLS §8 carries the dated rows; the phase tracker carries Round 12.
Date: 2026-09-06 · Author: session ccswitch · Scope: understand, not fix (per request)
The Systems → ccswitch sub-table showed morpheus@ Capped · 5h 108% (↻34m) · weekly 0% · 0 sessions, while the design had been described — including in my own Slack update ("gets out of its own way before hitting limits") — as if hitting a limit were impossible.
The switch tried four times, logged every attempt, and rescued nobody — because for the exact sessions that hit the wall, the rescue was structurally impossible. Two sessions (ba2937c8, 70690853) sat at Claude Code's native "Usage limit reached · continuing automatically at 10pm" wall, both still pinned to the capped morpheus token, both marked pending in restart-pending.json — and zero deferred-restart events fired all day. The switch was not lagging; it was defeated.
Why: ccswitch's move was detect at a turn boundary → mark pending → execute the restart at the session's next idle Stop. A session that hits its account's 5-hour cap does not reach a Stop — Claude Code's own client suspends the turn and enters "continue automatically at 10pm." That state is neither a completed turn nor an idle pane, so check_pending_restart never gets a clean Stop, cmd_deferred_restart never runs, the pending marker is overwritten by the next detection, and the process stays bound (§1: token bound at exec, immutable in a running process) to the dead account. The session waits hours for a reset while five accounts sit under 10%.
The guarantee was exactly inverted: the switch worked for healthy sessions that didn't need it and was inert for precisely the walled sessions that did — the rescue required the thing that had just broken (a normal turn boundary) to still be working.
(An earlier read said "no session is stuck" because the snapshot counts registered sessions and the walled ones had rotated their registration off morpheus — but the running processes never moved. The registration lied; the process table told the truth.)
EXEC_SNAPSHOT_FRESH_S = 900, snapshot every 600s): a session could be judging on 70% while the real number was 95%.A controller that detects late, samples discretely and acts deferred cannot hold a burst under its setpoint. 80% was a trigger, not a ceiling.
The spawn gate and score() steer sessions toward the account with the most headroom; morpheus had weekly = 0% — the emptiest on the board — so it was the most attractive destination. Several sessions piled on and collectively saturated its 5-hour window past 100% faster than the lagging switch could relocate them (weekly barely moved). The same mechanism that spreads load also concentrates it onto the current-best account between snapshots, and the 5h axis is the small, fast bucket that overshoots first.
Anthropic's unified-5h-utilization header reports a ratio that exceeds 1.0 when requests already in flight land after the window is nominally full — a soft, reported overage, not a wall we punched through. BEARER_SESSION_HOT = 0.97 is where the tool refuses to spawn onto an account.
check_pending_restart on the Stop hook → cmd_deferred_restart, which respawned via resume-by-session-id only when the pane read idle.Date: 2026-09-06 · Author: session ccswitch · Status: proposal (nothing built)
Goal (Gera's words): "the only goal is to make sure we don't get hit by rate limits by swapping at an appropriate time. If we need to stop to switch session, that's totally fine."
Today the fleet treats the wall as something to recover from: detect ≥80% at a turn boundary, mark the session pending, restart it at its next idle Stop. That design has two holes, and 2026-09-06 hit both: the meter it reads is up to 15 minutes stale, and the wall itself removes the Stop the rescue was waiting for. Two sessions sat at "continuing automatically at 10pm" all evening with five accounts idle at <10%.
The elegant version treats the 5-hour meter as something to never reach, and keeps one backstop for the wall. Every piece below reuses a primitive the fleet already runs — the snapshot job, the spawn gate, the wedge-healer's respawn, the transcript reader. Nothing new is invented; three knobs move.
Today: co.propflow.ccswitch-snapshot probes every token every 600s; a session judges its account on numbers up to 15 min old (EXEC_SNAPSHOT_FRESH_S = 900).
Change: the snapshot job runs every 60s, and on the fast ticks re-probes only the 5h axis of accounts that have live sessions (the full 7-day/pool/activity build stays at 600s). Six accounts × one 1-token probe per minute ≈ 6 requests/min for the whole fleet — nothing like the per-turn refresh cadence that revoked accounts (PITFALLS §6).
Effect: detection lag drops from ~15 min to ~1 min. Most of the 80→108 overshoot was this lag.
Today: one line, switch_at = 0.80, always polite: the move waits for the session's next idle Stop, and a busy session keeps spending on the hot account until then.
Change: keep 0.80 polite (no lost turn — right for the common case). Add switch_hard_at = 0.90: above it, the deferred helper restarts the session now, busy or not, using the exact primitive wedge-heal already uses — tmux respawn-pane -k + relaunch_cmd() → ccswitch exec -- claude --resume <sid> "continue". The respawn re-enters the spawn gate (server-probed, capped-last, headroom-ranked), so it cannot land back on the hot account; the primed "continue" resumes from the transcript.
Cost: the in-flight turn's partial output is lost (the transcript is not). Gera has accepted this trade explicitly.
Effect: with a 1-minute meter and a 90% hard line, a session leaves an account ~10 points before the wall. 108% becomes structurally unreachable by a fleet session.
The fact that makes this clean: Claude Code writes the wall into the session transcript the moment it happens:
{"type":"system","subtype":"informational","content":"Usage limit reached · continuing automatically at 10pm · esc or type to cancel"}
No pane scraping needed. wedge_detected() already reads exactly this tail (recent_assistant_turns → last 60KB of the transcript) to spot API-error loops and respawn them through the gate, capped at WEDGE_MAX_HEALS_PER_DAY = 3.
Change: a sibling predicate, walled_detected(tail): the newest system/informational entry says "Usage limit reached" and no assistant entry follows it. Evaluated by the snapshot job (which, after Move 1, already runs every 60s and already knows every live session from the reg stamps) — because the walled session's own hooks will never fire, the check has to live in a process that isn't the session. A hit → the same respawn as Move 2, same daily cap, logged to rotation.log as event: walled-heal with the account and destination in why.
Effect: anything that slips past Moves 1–2 (a burst faster than 60s, a session that was parked at 79% and woke into a busy account) is off the wall within a minute instead of at 10pm.
Today: the spawn gate ranks destinations by live registrations, then 7-day util. Registrations lag (renewed every 600s), so seven sessions leaving one hot account all see the same "0 sessions, 0% weekly" target and dogpile it — the exact shape that built morpheus's 108%.
Change: rank by util_5h + live_sessions × RESERVE (RESERVE ≈ 0.08 — one session's typical 5h burn). One-line change to the sort key in _exec_candidates; the k-th arrival sees the account as already k×8% fuller and picks the next one.
Effect: no thundering herd onto the emptiest account between snapshots.
autoContinueAtUsageLimit (user/managed setting, default ON for claude.ai subscriptions since v2.1.234): "wait in the open session and continue the task automatically after a claude.ai usage limit resets." At reset Claude Code sends itself a fixed "pick up where you stopped" prompt — on the same account. Nothing documented makes it exit or fail fast instead; /rate-limit-options is not in the docs.StopFailure hook fires "when the turn ends due to an API error" with matchers rate_limit, overloaded, authentication_failed, … — API-level errors only; no matcher for the claude.ai quota wall. Its output is ignored anyway. Not a trigger we can rely on.Notification hook has three quota matchers — quota_auto_resume_fired, quota_auto_resume_stale, quota_auto_resume_disabled — with undocumented payloads and timing. Their names suggest they fire around the resume (and "disabled" plausibly at the wall when auto-continue is off), but nothing on the mini has ever logged one, so this is unverified.system/informational "Usage limit reached" marker is real today and must be pinned by a test that fails loudly if the shape drifts (a drift guard, same as the fleet's other transcript readers).Consequence for the design: Moves 1–2 need nothing from Claude Code. Move 3's primary trigger stays the transcript marker (proven present, read by a process that isn't the walled session). The optional simplification — set autoContinueAtUsageLimit: false fleet-wide and hang ccswitch auto on Notification matcher quota_auto_resume_disabled for an in-session, zero-polling trigger — is worth a five-minute experiment the next time an account is at the wall (log the hook's stdin once); if it fires as its name suggests, it replaces the polling half of Move 3. The design does not depend on it.
/rate-limit-options juggling from outside). We don't argue with the wall; we respawn past it, which is the primitive the fleet already trusts for wedges. (If Claude Code turns out to offer a "exit on usage limit" setting, Move 3's detector can go — the pane's exit becomes the trigger and the existing respawn-on-death path takes over. Pending confirmation from the CLI docs; the design does not depend on it.)ccswitch.walled_detected fires, the respawn command is the gate-routed relaunch_cmd, the daily cap holds. Red-before against today's build.switch_hard_at low in a scratch config on one throwaway session, drive its account past it mid-turn, watch rotation.log record the interrupting restart and fleet-e2e-style verification that the respawned session completes a real turn on the new account.max(util_5h) across accounts in the bake never exceeds ~0.92, and walled-heal events are rare and never repeat on one session.Small. Move 1 is a plist interval plus a "fast tick" branch in cmd_snapshot; Move 2 is one constant, one config key, and dropping the busy-guard above the hard line; Move 3 is a ~30-line predicate beside wedge_detected plus a loop in the snapshot job; Move 4 is a sort key. Tests for each, one PITFALLS §8 row telling today's story. One PR, or two if the reviewer wants Move 3 separately.
CLAUDE_CODE_SESSION_ID; only its children (MCP servers, tool subprocesses) do, and argv holds an id only for resumed sessions. The first live drill found it; fixed as #69.cp of the local-bin plist kept its __HOME__ placeholder. Install through the README recipe.fleet-e2e reported the newest registration on the box, not its own session’s; fixed as local-bin #82.Residual, documented: a session that has spawned no child process yet has no readable id; every mini session carries two MCP children from boot.
PropFlow Docs