Always Shipping

Why PRs are not green, not merged, or not worked on at all while nobody is at the keyboard — and what rules replace the ones that cause that.

Decided 2026-09-17 — Fede adopted all of it except the settings registry (section 5, "Hygiene without a flag platform", first bullet), which is deferred: "too much for now".

TL;DR

• Nothing crashed overnight. The fleet stops because (1) a mechanical defect swallowed the wake-up events, (2) sessions end their turn with work still open, and (3) our own merge machinery has 13 ways to hold a green PR, and a sleeping session cannot answer any of them.
• Fix 1 (done today): wake-up waiters die with their session and can no longer hold a lock or eat an event.
• Fix 2 (rule): a session may not end a turn while it owns an open PR that is not armed for auto-merge. "End of night status" is not a state; the PR is either armed, merged, or the session is still working it.
• Fix 3 (rule): the review bot's "changes suggested" never holds a merge. Only a blocking finding holds. Suggestions become follow-up issues.
• Fix 4 (system): no new feature flags. Risky changes ship dark behind ONE of three existing switch kinds: the company live switch, a company setting on the customer page, or a property setting on the property page. Every switch is product configuration a customer or ops can see, with an owner and a review date. Nothing else is allowed.

1. What the 30 days show

CI speed is on its own page → Ship in 15 Minutes.

The machine ships. The problem is the night and the morning baseline, not throughput.

So "always shipping" is already mostly true. Three numbers explain the mornings that feel bad:

Read together: a third of PRs get held, a third need four or more rounds, and half the nights the fleet stops before those are cleared. Whatever is unfinished at the dead stretch is what Fede finds at 7 am. Seven open PRs this morning was a normal night, not a bad one.

2. Why last night specifically

Timeline (Denver time):

• 10:40 pm: last push by any session. Review rounds still open on five PRs.
• 10:40 pm → 5:20 am: zero pushes, reviews, or merges, fleet-wide. Verdicts kept arriving from GitHub and landed in the event file.
• 5:20 am: first session wakes on an unrelated notification and finds the night's verdicts as a "backlog".

Root cause A, mechanical. Each session is woken by a small waiter process that watches the GitHub event file. Waiters did not die when their turn ended. They lived up to 24 hours holding the session's "I already have a waiter" lock, so the next real waiter could not arm, and the stale one consumed each verdict and woke nobody. Found on the machine: 11 waiters whose Claude process had exited days earlier, some since September 10; the machine-wide duty lock held by a waiter from a turn that ended at 8:57 pm. Fixed today in the waiter script: a waiter exits within five minutes of its session ending, and a lock held by an orphan is taken over.

Root cause B, behavioral. Seven of the last fourteen nights have a three-hour-or-longer dead stretch, always after midnight. In 30 days, one in three "subagent finished" notifications (22,581 of 66,931) went unanswered for more than five minutes. The 25 longest silences were sessions that wrote a done-or-parked summary, or asked Fede a yes-or-hold question and stopped for a day. None were errors or limits.

Root cause C, structural. Thirteen distinct mechanisms can stop a green PR from merging without a person. Each is reasonable alone. Together they mean a PR needs a live session to clear it several times, and a sleeping session clears nothing.

3. The thirteen holds, and what to do with each

4. The rules that replace "end of night"

1. A session that owns an open PR does not end its turn until the PR is armed for auto-merge or merged. Blocking waits happen inside one long command, not by ending the turn.
2. A finished subagent is consumed by its parent within the same turn. A parent that has nothing to do with the result still says so and continues.
3. A yes-or-hold question to Fede never stops the session. It posts the one-line ask and keeps working everything that does not depend on it.
4. A sweep session wakes owners; it never takes over or splits a PR. A PR already under review merges as-is.
5. Morning baseline: zero open fede-authored PRs older than 12 hours that are not armed. Anything else is a defect the owning session explains in one line.

5. Risk without flags: the three-switch system

Fede's constraint: ship aggressively, keep customers stable, no proliferation of forgotten flags.

Rule: there are exactly three places a behavior can be switched, and they are all product configuration, not engineering flags.

What is banned: environment flags for behavior (*_ENABLED), code-level release toggles, allowlists in code, "temporary" gates. Today there are 5 env flags and 6 org/property settings. The 5 env flags get a removal PR each or become a company setting.

How a risky change ships:

1. It ships merged and deployed the same day, dark, meaning: off for every company by default because it is a company setting whose default is off.
2. It is turned on at a test company (Fairhaven, Willows) by the session, with a proof on the tracker.
3. Fede turns it on per customer from the customer page. That is the only "flag" anyone ever manages, and it is visible on the page the team already reads.

Hygiene without a flag platform:

• A settings registry (one file) lists every company and property setting: name, default, owner, date added, "review by" date, the page that shows it. The build fails if a setting is read in code and missing from the registry.
• The customer page shows every setting for that company, with its value. Nothing hidden.
• Monthly: any setting that is on for every customer becomes the default and its switch is removed. Any setting off for every customer for 90 days is questioned.

6. Options for Fede

Decision: A, minus the settings registry.

1. A. Adopt sections 3–5 as written (recommended). Dangerous-diff floor narrowed, 🟡 no longer holds, three-switch system, settings registry.
2. B. Adopt 3–4 only; keep flags as they are. Fixes the overnight stalls, leaves the flag question for later.
3. C. Keep holds as they are; only fix the waiter defect and the end-of-night rule. Least change, most mornings like today.

7. What was done today

• Waiter defect fixed in the wake script; 11 orphaned waiters killed.
• Trinity's prompt: "ship it yourself" replaced with "conduit into sessions"; PR open on agent-smith.
• Sweep ledger and every live session carry the no-takeover, no-split rulings.
• Reminders opt-in PR: last two review findings fixed, auto-merge armed.

PR flow, 2026-09-15 to 09-28: why WIP is high and merge-to-live is slow

Read-only pull from GitHub + Vercel + git history, 2026-09-28. Full numbers and every PR in the sample: appendix below.

Last week vs. the week before

Review is not the problem — last week beat the prior week on every review measure while shipping 34% more work. What got slower is the last mile: once a PR merges, it now waits for a 30-minute batch window before it goes live, so a typical PR now takes about 1.4 hours from opening to live, up from 1.0 hour.

Where the time goes (two weeks combined, PR-hours)

The single biggest sink, parked-green PRs, is only 40 PRs — all in Gera's lane, none armed for auto-merge, 4 carrying a hold-for-review label.

Are 50+ comments on a PR normal, or a bug?

Mostly normal. A PR with 50+ comments has, by design, about 10 review rounds — each round writes a run summary, a verdict summary, and inline findings, and the ≥50-comment PRs are big (median 1,570 lines changed). Reading a sample of what triggered the blocking reviews on these PRs: about 60% were real code defects worth catching. Roughly 20% was avoidable noise from the review system itself — mainly the plan reviewer posting a brand-new comment on every push instead of updating one (13% of all comments on these PRs, concentrated in a spike from 09-16 to 09-19 while its red verdict held merges), plus sessions pinging for a review that was already running (another ~7% together with the "already running" replies). One genuine review-system bug sat in the window and was fixed on 09-18: a blocking review that quoted an earlier green verdict was being misread as an approval.

What we couldn't measure

• Time from starting the work to opening the PR — branches are committed and opened together, so there's no visibility before the PR exists.
• How long a PR sat armed before it merged — the "auto-merge armed" event came back empty from GitHub's API.
• Conflict (DIRTY) history — no event records it; a same-branch merge-from-main commit was used as a rough stand-in.
• Backend (Lambda) deploy times — only the web app's production deploys were measured.
• 71 PRs merged last week have no matching deploy yet, mostly from the last few hours of 09-28.

Proposed — pending review

1. Refuse a PR over ~600 lines until it's split. Big PRs review much worse (600+ lines average 3.8–5.8 rounds vs. 1.8 for 100–300 lines). Estimated effect: about 18% fewer review rounds across the two weeks, and the worst-case (90th percentile) open→merge time drops from about 5 hours toward about 4.
2. Clear or arm Gera's parked PRs the same day, through Smith. Targets the 4,656 PR-hours of green-but-unmerged work above. Estimated effect: open PRs drop from about 92 today to about 50, by closing or landing the ones over a week old and the ones on hold.
3. Block re-review until the flagged lines actually change, and make the plan reviewer edit one comment instead of posting a new one on every push. Estimated effect: removes the roughly 28% of blocking rounds that just repeat an unfixed finding from the previous round, and cuts comment volume on big PRs by about 20%.

Not ranked, but worth a decision: the 30-minute deploy batch costs about 25 minutes median per PR (merge→live went from 13 to 37 minutes median). Deploying immediately whenever nothing has shipped in the last 30 minutes would win most of that back without losing the batching benefit.

PR cycle retro, 2026-09-28

Read-only pull, 2026-09-28 23:50 UTC, scope: 409 fede-propflow PRs opened 09-25–28 (372 merged, 22 open, 15 closed), every verdict/comment/check/deploy event, vs. the prior week (09-18–24, 812 PRs). Full report: scratchpad/pr-cycle-retro-2026-09-28.md.

Headline numbers

The typical PR is faster than last week. What Fede sees is the tail: 50 PRs (12%) took more than 4 hours, and they hold most of the time — 823 PR-hours spent open in the window, 246 (30%) of it PRs sitting as drafts.

Top 3 causes of the tail

1. Parked as a draft waiting for Fede's go — 15 PRs, 277 PR-hours. Merging would have turned something on at a customer (self-guided tours, staff texting, a stage chain), so the session drafted it instead of cutting a default-off switch first.
2. Review-round fix loops — 12 PRs, 83 PR-hours of real defects, concentrated in big PRs (PRs over 1,000 lines were 7% of all PRs but produced 38% of all 🔴 findings and 45% of fix-loop hours).
3. Conflicts / re-merge with main after green — 11 PRs, 72 PR-hours; the guard's "main moved" hold fires even when the only change is a row added to a drift-test inventory.

The 8 changes

Estimates scale the 4-day window by 1.75 for a weekly figure; they overlap, so do not add them up.

Dark-features register (minimal)

Problem: work that merges dark gets forgotten — the first-email engine sat unfinished for days, and several weekend features are dark with no owner tracking the turn-on.

• Source: no new registry. The company/property settings that already default off, found via a one-line PR-body field review:self requires whenever a diff adds a setting key: Dark: <setting key> | feature: <plain words> | proof: <Willows/Fairhaven link or "pending"> | turn-on: Fede.
• Nightly script ~/.claude/scripts/dark-register.sh: one gh search call for merged PRs (last 60 days) carrying a Dark: line, then a read-only admin settings read per company for that key. Writes a JSON block into this page. No build check, no settings registry (Fede, 09-17: "too much for now").
• Columns: feature, setting key, merged date, companies on/off, proof link, owner lane, turn-on gate (always Fede), days dark.
• Where: one "Dark features" section on this Decisions page, not a new page.
• Staleness rule: dark >7 days with no proof link renders as "prove it or delete it", goes to the owner lane. Proven and still dark >7 days goes on Fede's morning list as one line: "turn on at <company>? yes / hold." Dark >30 days gets a removal PR (prove-or-remove, 09-24 precedent).
• Kept current without manual work: the PR-body field is enforced at push time, the nightly script owns the page, the morning list reads the page. Nothing is typed by hand.

Feature inventory, Sep 15–29 2026 — what's on or off, per company

Fede, 2026-09-29: "we built a lot of stuff in the last two weeks and I don't know if everything's on or off." This fills the register above with real data. Method: grouped ~3,570 merged PRs (2026-09-15 to 2026-09-29) into features, found each feature's setting key in the settings registry (src/lib/domain/portfolio/registry/keys.ts), then read the LIVE value straight from production DynamoDB on 2026-09-29 for the four companies below. No customer names, no phone numbers.

Live values read 2026-09-29 by targeted get-item/query calls against propflow-prod (org PROFILE row + one sample property's LEASING_SETTINGS row per company); no full-table scans. Org ids: Western Slope org_c42d9150…, JP&Co org_jpco, Situs Group org_a3376147…, Willows org_sandbox.

Row for #20637 (staff-asked prospect text) added by hand 2026-10-17; it is not part of the 09-29 live read above.

Not merged yet (Fede-authored PRs open >24h, as of 2026-09-29)

• #12871 — texts the property staff to ask for the lockbox code when someone books a self-guided tour (~46h old, marked "DRAFT, do not arm" in the title).
• #12879 — puts the lockbox code in the prospect's reminder text and texts staff when the self-guided tour starts (~45h old).
• #12892 — when staff reply with the lockbox code, it saves onto the tour (~45h old, marked "DRAFT" in the title).
• #12910 — lets a prospect create, confirm and reschedule a self-guided tour (~44h old).

#12871 and #12910 armed for auto-merge; #12879 and #12892 stacked behind them, armed as each parent merges. All four are the self-guided-tours chain; the feature above reads OFF everywhere until they land.

No hard WIP cap

Not recommended — the data does not support it. Fede rejected one on 09-19, and the WIP problem is made of the drafts in change 1, not of too many live PRs. Enforcing change 1 plus the existing 3-per-session soft limit is enough. Nor a round cap that merges on 🟡: 🟡 already never holds, and 73% of 🔴 findings in the window were real defects.

Every PR in 09-15..09-28 with 30+ comments (244) — click to expand

Red-cause labels are from a keyword classifier (rough: 56% agreement with a model-read sample). "Machine-written" excludes only the three rows of human/session prose from the comment-source breakdown above.

PR	author	comments	rounds	red rounds	red causes	repeat-blocker reds	machine-written	open→merge h	lines
#9425	fede	156	16	9	defect 6, stale 2, process 1	1	76%	13.5	2684
#9721	fede	122	19	13	defect 10, proof 1, process 1, reversal 1	1	75%	14.5	3030
#9782	fede	118	22	3	process 2, reversal 1	1	99%	19.4	3624
#8962	fede	96	12	5	defect 5	2	83%	15.7	2237
#9426	fede	89	15	12	defect 7, proof 3, process 2	1	85%	8.8	3994
#12017	fede	87	15	9	proof 8, defect 1	4	78%	8.1	2249
#9633	fede	82	8	6	defect 4, proof 1, infra 1	2	78%	3.3	3114
#9635	fede	81	10	9	defect 5, proof 3, process 1	4	74%	4.2	1337
#10775	fede	81	14	8	defect 8	4	94%	24.2	1490
#9427	fede	78	13	3	process 2, stale 1	1	95%	17.6	904
#9933	fede	78	13	10	defect 7, stale 2, infra 1	2	91%	4.8	1705
#9105	fede	78	11	4	process 2, defect 2	2	82%	12.5	3608
#9783	fede	77	11	9	defect 8, process 1	6	88%	19.5	1774
#9990	gera	76	15	13	defect 13	7	84%	30.4	8432
#12027	fede	75	11	6	defect 3, proof 2, reversal 1	4	69%	5.5	2354
#9921	fede	73	10	8	defect 5, process 2, proof 1	5	89%	4.0	1973
#9203	fede	73	15	5	defect 5	1	84%	7.8	1370
#9046	fede	72	15	8	defect 6, process 1, reversal 1	5	99%	14.9	1401
#9956	fede	72	12	9	proof 2, ci-red 2, defect 2, stale 2, process 1	4	88%	5.0	8935
#8649	fede	71	13	8	defect 5, proof 2, process 1	2	86%	8.9	1648
#9149	gera	70	10	3	defect 3	0	77%	4.3	1010
#10134	fede	68	9	7	defect 4, process 3	1	94%	2.8	1722
#11865	gera	67	22	0	-	0	88%	14.1	1584
#9994	gera	66	13	12	defect 6, rule 4, proof 2	9	67%	29.7	4000
#12977	fede	65	9	7	defect 6, rule 1	5	88%	8.3	2015
#9497	fede	64	10	8	defect 6, proof 1, stale 1	6	88%	6.1	683
#9844	gera	64	10	1	rule 1	0	78%	3.4	979
#8523	gera	64	14	3	defect 2, proof 1	1	89%	23.7	2430
#13187	fede	63	13	12	defect 11, rule 1	7	100%	open	1344
#12791	gera	62	12	4	defect 3, process 1	1	100%	32.5	1533
#9272	fede	62	12	6	defect 3, reversal 2, ci-red 1	0	100%	4.9	799
#9484	fede	61	7	3	stale 2, process 1	2	87%	10.2	294
#9653	fede	61	10	8	defect 8	1	87%	6.4	2603
#9435	fede	60	13	5	defect 2, proof 2, ci-red 1	3	92%	12.4	411
#12974	gera	60	9	6	defect 4, proof 2	0	87%	4.0	1518
#10131	fede	59	10	7	defect 4, infra 1, proof 1, reversal 1	3	85%	4.6	304
#9664	fede	58	5	4	defect 3, ci-red 1	2	71%	1.3	3398
#11115	fede	57	16	10	defect 8, process 2	2	100%	6.6	2429
#9705	fede	57	7	3	defect 2, stale 1	0	70%	1.9	764
#8670	fede	57	6	5	defect 4, proof 1	2	89%	10.3	848
#10769	fede	56	10	5	defect 1, process 1, reversal 1, proof 1, stale 1	1	93%	8.4	667
#9928	fede	56	7	6	process 4, defect 2	5	91%	1.4	804
#9961	fede	54	9	9	defect 7, process 1, rule 1	2	81%	closed	2404
#9553	fede	53	10	4	defect 3, reversal 1	1	96%	6.3	251
#10015	fede	53	5	3	defect 2, proof 1	2	83%	1.4	1713
#9234	fede	52	8	2	process 2	0	98%	4.5	1567
#9511	fede	52	9	5	defect 5	3	90%	7.5	208
#9492	fede	51	10	7	defect 6, process 1	2	86%	3.3	1574
#10309	fede	50	11	4	defect 3, process 1	1	84%	6.0	954
#9053	gera	50	11	2	stale 1, proof 1	0	88%	closed	794
#13036	fede	50	11	8	defect 7, reversal 1	3	86%	open	1449
#11872	fede	50	6	5	defect 5	2	100%	5.2	3144
#9013	fede	50	7	0	-	0	100%	8.4	528
#9070	gera	50	13	0	-	0	98%	3.3	742
#9658	fede	49	9	7	process 7	4	98%	3.2	1398
#12856	fede	49	11	9	defect 6, process 3	4	94%	6.4	3471
#12008	gera	49	11	6	defect 3, proof 2, reversal 1	0	94%	4.9	2293
#8165	gera	49	8	3	? 2, proof 1	1	71%	113.3	1056
#10836	gera	48	11	7	defect 3, rule 2, ci-red 2	4	73%	3.3	2898
#9848	gera	48	11	1	defect 1	0	92%	11.5	1634
#9720	fede	48	6	4	defect 4	3	96%	2.2	3022
#12307	fede	48	9	8	defect 7, proof 1	0	83%	50.2	1330
#8166	gera	48	8	0	-	0	83%	90.8	1102
#9622	gera	47	11	6	defect 4, proof 2	1	79%	14.3	1093
#8978	fede	47	6	2	defect 2	0	98%	1.4	926
#10049	fede	47	9	7	process 3, defect 2, proof 2	1	98%	4.7	974
#9081	gera	46	7	6	defect 5, process 1	0	78%	34.1	1667
#10604	fede	45	7	5	defect 5	0	87%	6.8	994
#10008	fede	45	5	3	process 3	3	93%	1.9	746
#8986	fede	45	6	3	defect 1, process 1, reversal 1	2	93%	13.6	683
#7942	gera	45	8	0	-	0	64%	97.4	791
#8945	gera	45	6	2	process 1, infra 1	1	80%	2.0	1320
#9023	fede	44	10	1	defect 1	0	91%	13.6	666
#9244	fede	44	9	4	defect 4	0	86%	2.1	172
#9948	fede	44	5	3	defect 2, process 1	1	89%	1.4	2683
#8977	fede	44	7	3	defect 2, proof 1	2	89%	12.0	276
#9756	gera	44	6	2	process 2	1	98%	2.0	1858
#9452	fede	43	9	2	ci-red 2	0	91%	11.9	278
#8893	fede	43	6	3	defect 3	0	86%	2.7	1229
#9039	fede	43	10	7	defect 6, process 1	3	100%	14.6	1072
#8813	gera	43	6	3	? 1, defect 1, process 1	0	79%	4.3	1300
#9750	fede	43	6	1	process 1	0	84%	3.4	883
#12286	fede	43	10	7	defect 3, process 2, proof 2	2	100%	4.6	1367
#12305	fede	43	6	3	defect 1, proof 1, reversal 1	1	100%	6.3	1471
#9477	fede	43	6	4	process 2, defect 1, proof 1	2	81%	4.9	118
#11414	fede	43	11	8	process 5, defect 1, infra 1, stale 1	6	98%	12.8	4308
#10201	fede	43	8	4	defect 4	0	88%	2.1	1072
#10241	gera	43	1	0	-	0	100%	5.5	22
#9402	gera	42	13	3	proof 2, defect 1	1	95%	18.7	1795
#9487	fede	42	7	5	process 3, proof 2	3	90%	1.9	550
#9576	gera	42	7	5	defect 5	3	95%	closed	2030
#9509	gera	42	8	0	-	0	98%	14.3	557
#12863	fede	42	9	5	ci-red 2, defect 2, stale 1	0	83%	open	1246
#10157	fede	42	8	0	-	0	95%	7.2	1635
#12149	fede	42	6	2	defect 1, ci-red 1	1	81%	closed	997
#8674	fede	42	6	3	defect 2, stale 1	0	86%	8.7	827
#11985	fede	42	8	4	proof 2, defect 2	2	76%	2.4	1348
#9543	fede	41	8	4	defect 3, proof 1	2	100%	6.5	675
#12022	fede	41	5	4	defect 2, proof 2	3	76%	1.5	964
#12076	gera	41	8	3	defect 3	1	90%	5.5	1295
#8937	gera	41	7	1	proof 1	0	85%	2.4	736
#9167	gera	41	7	4	defect 4	3	90%	6.3	1149
#9979	gera	41	6	2	defect 1, process 1	2	66%	3.4	806
#11765	fede	41	7	3	proof 2, defect 1	2	80%	3.3	1594
#11292	fede	41	8	8	defect 6, proof 2	4	95%	closed	3702
#9860	fede	41	7	3	defect 3	2	88%	3.0	821
#9771	fede	41	5	1	defect 1	1	83%	11.0	902
#10835	fede	40	8	4	defect 3, stale 1	2	88%	4.8	998
#8888	fede	40	4	3	defect 3	1	92%	3.1	590
#10064	fede	40	8	3	defect 3	0	98%	19.2	1271
#8844	fede	40	6	2	defect 2	1	80%	2.3	435
#10560	fede	40	9	3	defect 3	1	80%	5.7	93
#10567	fede	40	5	3	defect 2, process 1	1	78%	2.6	1634
#10016	fede	40	4	3	defect 3	1	100%	3.9	1258
#9180	gera	40	9	0	-	0	88%	4.8	1283
#9753	gera	40	7	1	process 1	1	78%	7.4	1444
#10316	fede	39	8	5	defect 4, rule 1	2	100%	3.5	1858
#9455	fede	39	7	5	defect 3, proof 2	2	87%	4.4	406
#10822	fede	39	8	6	proof 3, process 2, ci-red 1	4	92%	21.0	1936
#10877	fede	39	6	5	defect 4, rule 1	1	54%	2.6	1164
#8782	gera	39	7	0	-	0	82%	3.5	1272
#9779	fede	39	5	2	defect 2	1	85%	8.6	474
#9885	fede	39	7	5	defect 2, proof 2, ci-red 1	3	82%	2.5	1201
#10534	fede	39	5	3	defect 1, reversal 1, process 1	0	85%	1.2	1094
#9089	gera	39	7	1	process 1	0	100%	1.9	158
#9109	gera	39	7	0	-	0	100%	4.0	42
#8669	fede	39	6	4	defect 3, process 1	1	90%	closed	864
#9304	gera	39	8	7	defect 3, process 2, proof 2	4	77%	2.0	731
#10397	gera	38	8	1	reversal 1	1	82%	3.0	743
#10694	fede	38	5	2	defect 1, reversal 1	0	92%	3.1	1696
#9030	gera	38	7	0	-	0	74%	17.9	1374
#8852	fede	38	7	3	stale 1, process 1, proof 1	1	89%	2.8	862
#10010	fede	38	7	4	defect 4	2	92%	13.0	2479
#10166	gera	38	4	0	-	0	95%	3.3	12
#9737	fede	38	6	3	defect 3	0	79%	1.9	322
#9282	fede	38	6	4	defect 4	1	89%	2.2	194
#10904	fede	38	7	6	defect 5, process 1	1	84%	2.3	1341
#8780	gera	38	5	1	defect 1	0	66%	4.0	446
#8999	fede	38	5	1	defect 1	0	84%	1.1	556
#9784	fede	38	7	3	defect 2, reversal 1	2	87%	8.6	1534
#10388	fede	37	7	4	defect 3, reversal 1	0	92%	2.5	1296
#10440	gera	37	12	7	proof 3, process 2, defect 2	4	76%	8.9	1286
#9479	fede	37	5	1	process 1	0	78%	6.4	312
#9962	fede	37	7	3	defect 3	0	95%	13.4	2267
#13185	fede	37	8	2	proof 2	1	100%	open	1239
#8944	gera	37	6	2	defect 2	0	84%	0.8	650
#10141	fede	37	7	6	defect 3, proof 2, rule 1	0	89%	2.5	1617
#11901	fede	37	10	7	defect 3, ci-red 3, stale 1	2	100%	2.2	1992
#10882	fede	37	8	4	defect 4	2	92%	4.1	1814
#10418	gera	36	8	5	process 3, defect 2	2	94%	91.6	1563
#8864	gera	36	6	4	defect 2, stale 2	1	78%	1.1	154
#9062	gera	36	9	1	reversal 1	0	89%	14.5	1091
#9125	gera	36	8	6	defect 5, infra 1	4	92%	1.8	665
#8309	gera	36	4	1	defect 1	0	86%	52.2	987
#11790	fede	36	12	4	process 2, ci-red 1, proof 1	2	97%	7.8	982
#8702	gera	36	3	1	defect 1	0	97%	0.6	602
#9459	fede	36	6	2	defect 2	0	86%	5.4	579
#9475	fede	36	6	5	proof 5	2	78%	6.2	428
#9874	fede	36	6	4	process 3, stale 1	1	92%	4.0	5441
#9308	gera	36	4	3	defect 3	0	75%	1.2	507
#9758	gera	36	10	0	-	0	94%	3.4	1535
#12679	fede	35	7	5	defect 4, proof 1	4	100%	14.7	2126
#11913	fede	35	7	6	defect 5, rule 1	1	100%	3.5	2153
#9236	fede	35	8	5	defect 4, proof 1	2	100%	2.2	2797
#8796	gera	35	5	1	process 1	0	89%	2.5	799
#9890	fede	35	4	2	defect 2	0	97%	1.8	752
#8736	gera	35	7	1	proof 1	0	94%	60.3	2135
#12289	fede	35	10	6	defect 4, process 2	2	74%	3.0	264
#10025	fede	35	5	3	defect 3	2	97%	3.1	1043
#9884	fede	35	4	2	defect 1, ci-red 1	0	91%	8.9	641
#7366	gera	35	8	1	defect 1	0	71%	172.1	762
#9657	fede	34	1	1	defect 1	0	68%	0.5	1582
#10713	fede	34	7	4	rule 2, proof 1, defect 1	0	100%	9.7	1061
#9527	gera	34	8	1	defect 1	0	94%	175.2	2081
#9524	fede	34	6	3	process 2, stale 1	0	97%	2.4	1256
#9057	gera	34	8	0	-	0	100%	2.6	1410
#10756	fede	34	6	2	process 1, defect 1	0	88%	5.9	2052
#9583	fede	34	5	2	defect 2	0	91%	4.7	315
#9457	fede	34	7	3	defect 3	0	100%	3.7	502
#8637	fede	34	7	2	defect 1, stale 1	1	94%	29.2	845
#11227	fede	33	6	4	stale 2, process 1, defect 1	2	85%	11.0	1808
#9204	fede	33	6	4	process 2, proof 1, defect 1	1	97%	1.8	556
#9235	fede	33	6	2	defect 2	0	91%	closed	657
#12855	fede	33	7	3	defect 1, proof 1, ci-red 1	1	97%	open	1146
#10706	fede	33	6	4	defect 4	2	100%	9.8	1410
#9966	fede	33	4	2	defect 1, proof 1	0	94%	6.5	1451
#10012	fede	33	3	2	defect 2	0	100%	3.1	841
#9833	gera	33	6	2	process 2	0	85%	1.2	1235
#10752	fede	33	8	5	defect 3, process 2	2	100%	1.9	404
#10004	gera	33	6	3	defect 1, process 1, reversal 1	0	67%	29.2	1190
#8738	gera	33	3	0	-	0	97%	3.2	313
#12455	fede	33	9	8	defect 6, proof 2	4	97%	2.8	1102
#9088	gera	33	7	2	defect 2	1	97%	1.5	730
#9106	fede	33	5	3	process 3	2	82%	2.5	1179
#8572	fede	33	7	3	process 1, infra 1, defect 1	1	79%	4.9	401
#9201	fede	33	6	4	defect 2, proof 2	2	88%	2.4	1203
#11075	fede	32	6	4	defect 4	1	88%	22.6	322
#10281	fede	32	7	4	defect 3, infra 1	1	100%	4.4	913
#9443	gera	32	5	1	process 1	0	84%	2.3	1028
#8855	fede	32	5	2	defect 2	0	88%	1.6	1673
#12038	fede	32	7	6	defect 6	2	100%	10.0	2669
#8853	fede	32	5	1	defect 1	1	81%	1.8	911
#12643	fede	32	7	2	proof 2	0	88%	7.8	855
#10167	fede	32	7	3	proof 2, defect 1	1	97%	3.2	1495
#9168	gera	32	5	2	defect 2	0	75%	2.8	1127
#8560	fede	32	7	2	rule 1, ci-red 1	1	72%	6.1	1080
#9891	fede	32	5	2	process 1, defect 1	1	91%	2.3	699
#10572	fede	32	5	2	defect 2	1	94%	1.9	1169
#10031	fede	32	5	4	defect 4	0	100%	2.2	1405
#11943	fede	32	5	4	defect 4	1	81%	1.0	795
#9291	gera	32	6	4	defect 4	0	91%	1.6	798
#9179	gera	32	9	2	defect 1, ci-red 1	0	91%	4.6	869
#11160	fede	31	11	7	defect 6, proof 1	3	94%	10.8	1139
#9644	fede	31	6	2	defect 2	1	97%	1.4	630
#9717	gera	31	9	0	-	0	65%	41.8	885
#9054	gera	31	5	4	process 3, defect 1	1	81%	14.7	382
#9968	fede	31	6	4	defect 4	0	97%	2.1	603
#10130	fede	31	6	4	defect 2, reversal 1, ci-red 1	1	94%	1.6	963
#8900	gera	31	6	3	defect 3	2	87%	1.2	809
#12536	fede	31	5	4	rule 1, proof 1, defect 1, process 1	0	87%	2.3	585
#9852	gera	31	8	1	process 1	1	94%	10.2	2320
#11896	fede	31	5	2	defect 2	1	84%	1.2	1276
#10933	fede	31	7	5	defect 4, stale 1	2	94%	2.6	1140
#9579	fede	31	6	3	process 2, defect 1	1	100%	3.4	191
#11372	fede	31	5	4	process 2, defect 2	1	84%	3.5	889
#10240	fede	31	4	1	defect 1	1	84%	6.7	770
#9883	fede	31	5	3	process 2, defect 1	2	90%	1.5	1352
#8964	fede	31	4	1	proof 1	0	87%	2.0	523
#9226	fede	30	7	2	stale 2	0	97%	1.7	540
#10255	fede	30	5	4	defect 2, proof 2	0	83%	closed	972
#10794	gera	30	4	3	defect 3	1	83%	3.2	1161
#13192	fede	30	7	1	defect 1	1	100%	2.9	564
#13245	fede	30	4	1	defect 1	0	100%	open	1557
#8925	fede	30	3	3	defect 3	0	83%	closed	1209
#12595	gera	30	10	3	ci-red 3	1	90%	4.2	1133
#12648	fede	30	4	2	defect 2	1	90%	10.4	102
#11941	fede	30	7	3	defect 3	1	97%	6.5	1630
#8431	gera	30	7	3	proof 2, defect 1	0	80%	33.6	538
#12255	fede	30	8	3	defect 3	2	77%	3.0	283
#12268	fede	30	7	3	defect 2, process 1	1	73%	3.2	415
#9084	gera	30	6	5	defect 2, proof 2, process 1	2	97%	1.0	591
#11283	fede	30	5	5	defect 4, ci-red 1	4	93%	closed	2591
#10253	fede	30	7	3	defect 3	0	90%	6.0	963
#9285	fede	30	5	2	defect 1, process 1	1	100%	1.6	1182

Retro 2026-09-29 — "finish all WIP tonight" ended with 15 open PRs

Sources: verbatim accounts from sessions 000, 001, 004, 006, and the orchestrator (003), collected the same morning; the PR timeline pull for 2026-09-28T18:00Z–2026-09-29T14:30Z (100 fede-authored PRs); the orchestrator's overnight log. Full accounts: scratchpad/retro-session-accounts-2026-09-29.md. One data-collection error: the PR-timeline pull reports zero 🔴 findings in the window — the session accounts describe dozens of 🔴 rounds, so review-round and red-main counts below come from the pull, but "zero reds" is not used anywhere in this retro.

Fede's question, verbatim: "every time we say no more work in progress, finish tonight, then I come back and there's dozens of open pull requests and the sessions seem to be asleep. What is going on?" Sessions were awake and working almost the whole night — the accounts show pushes, rebases, and merges into the early morning. The open list built up from four separate mechanisms, not from anyone stopping.

What Fede saw at 07:50 MDT

Ages are hours-open readings from the PR-timeline pull (collection window ends 14:30Z), so a few merged shortly after Fede's 07:50 look. The chain PRs predate the pull's window; their age is described, not measured.

Root causes, ranked by hours lost

1. A turn ended "waiting on PR events" while a DIRTY or disarmed PR fires no webhook. Biggest single loss. Session 000 went idle 166 minutes (00:04–02:50Z) and 293 minutes (09:08–14:01Z) — almost 7.7 hours — both times with only the event pipe registered while its PRs sat DIRTY or unarmed. Session 001 registered a five-minute GitHub poller that the interval guard killed (exit 144); a red check at ~07:45Z wasn't read until ~13:55Z, a six-hour gap.
2. Red main cascade. Mechanism: repo-wide ratchet tests, branch protection set to non-strict, an affected-only test path on ordinary PRs, and every red main run disarms every open PR. Overnight reds: #13566 (team_alert), #13601 (drift-row import guards), #13609 (NONE_RATCHET), the #13690→#13704 pair, the #13704→#13717 pair. Session 004 separately logged about 50 main-red runs across 09-28 daytime (10:01–20:27Z).
3. A pre-push control pins a line number. The "absent-means-provenance" mutation control asserts the settings resolver at a fixed line. Every merged setting moves that line, and then every push in the repo is refused until someone re-pins it: twice on 2026-09-29 (#13602 moved it, #13640 re-pinned 08:30–09:00Z; another setting moved it 107→110 at ~14:40Z, Gera's #13754 derives the line from the key list instead). Session 000's #13577 sat ready but unpushable in between. Same family as the ratchets above: a rule keyed to a line number or a count breaks on every unrelated merge.
4. Green PRs sat BLOCKED on stale check-runs. Two shapes, ~8 hours between them. #13600 (session 001) was armed and green for six hours while GitHub kept a carried-forward review=failure check-run from 07:45Z; a later review=success in a different check suite never overrode it, and a review rerun re-stamped success without clearing it. #13714 (session 003): a PR whose affected-test selection exceeded the 200-file cap gets "Unit Tests (decision)" left in progress until review, the required "Unit Tests" verdict posts only on a review run, and the review bot skipped the new head because GitHub had moved an earlier APPROVE forward. Cures today: gh pr update-branch for a fresh head, and dismissing the stale bot approval. Both are handed to Smith on the bridge; the fix is to compare against the original commit when deciding whether a review still stands, and to post one required verdict per head.
5. The load guard refused local test runs. Spotlight's mds/mdworker processes sat at 200–350% CPU indexing 16 full lane clones (not worktrees), so the guard blocked builders' own test commands and CI became the first run of the tests. Each defect that a two-minute local run would have caught instead cost a full CI-plus-review round: ~1.5h for session 001, ~2h for 006, two refusals logged by 000.
6. Review rounds ran long. #13511 eleven rounds (fixture drift, a judge-error control that exited 0, a too-loose rubric, a leaked base-snapshot street, guard-pin dodges, main's ratchet moving mid-flight); #13629 eleven-plus (two 🔴 on the same release path, redesigned); #13611, #13577, #13548 six rounds each (docblocks plus an unsourced count; a scratchpad file two PRs both wrote; green then DIRTY).
7. Stacked PRs got auto-closed. Arming a parent with --delete-branch auto-closed its stacked child twice — #13448 and #13635 — and a closed PR can't retarget, so each needed a brand-new PR (#13701, #13728) and a fresh review from round one. #13614/#13615 were dropped the same way.
8. The #12855 chain sat on the 1,000-line gate. 37 hours waiting on an admin merge, with three PRs (#12871, #12879, #12892, #12910) stacked behind it and unable to move until it does.
9. The orchestrator swept only its own PRs. It never ran gh pr list --author fede-propflow, so eight PRs opened by other sessions aged 6–10 hours unseen. It reported "everything merged except two" at 13:5xZ — wrong, for that reason.
10. CI ran on the expensive path for a week. CI_RUNS_ON sat at 0 for seven days (09-21–09-28), pushing every run onto GitHub-hosted runners instead of self-hosted; session 004's billing read shows $533 / $260 / $1,026 gross on 09-26/27/28.

What changed already this morning

• Load guard refactored to wait-then-allow: tests are never flatly refused (~/.claude/scripts/load-guard.sh, hook timeout raised to 600s).
• Local test runs in lane clones with a shared node_modules: PR #13739 open and armed (before: every local vitest run refused, "CI is the gate"; after: same run passes locally). Accepts only a symlink to another checkout of the same repo with a byte-identical lockfile and identical in-repo packages.
• A red-main coordination question is posted to Smith on #engineering-bridge.
• All 15 PRs above now have a named owner.
• Retro accounts collected from every session, feeding this section.

Proposals

Numbers

Source: PR-timeline pull, 2026-09-28T18:00Z–2026-09-29T14:30Z, 100 fede-authored PRs.