0041 — CI history is kept, so a trend can be read; a silence is a gap, never a zero

• Status: Accepted. Built and live 2026-10-03: ~/.local/bin/ci-factory-flow writes the history (gera-propflow/local-bin#405); the scorecard strip (propflow-docs#295, #297) and the reds chart (#296) are on docs.propflowai.co/a/ci-factory-floor. First live-path write 23:45Z: history ok flow=6 gaps=1 days=4 reds=76.
• Date: 2026-10-03
• Deciders: Gera green-lit the build on the proposal at docs.propflowai.co/a/ci-quality-grade-2026-10 (five measures with their own targets, no single score; "reds" and "handled" defined there). The shapes below were chosen by the worker that built them, per ADR-0004.
• Systems: ci-factory-mq (its sibling ci-factory-flow), propflow-docs functions/api/factory.js and the /a/ci-factory-floor page.

The problem

Every number the CI factory pages show is about now. mq.json is the queue this minute, floor.json the run testing main this minute, flow.json one rolling 24 hours, overwritten every 6 hours (it keeps 7 days of headline rows, nothing older, and nothing per day about main). Nothing kept yesterday, so "is CI getting better, or did we just have a good day?" had no answer, and the proposal's five measures had no trend to be read against. Worse, the one thing a dead collector produces is nothing, and a page that draws nothing as zero reds would read a broken instrument as a clean day (ADR-0009).

The decision

1. One history object, written by the pass that already has the numbers. After flow ok, ci-factory-flow also writes R2 propflow-podcasts/ci-factory/history.json: version, generatedAt, and three arrays, 60 days deep. One object, because /api/factory reads it in one fetch and passes a trimmed view to the page as snap.ciHistory (14 days, 48 h of reds) — not snap.history, which the CI Factory collector's own factory.json already carries (725 points of station counts the full page's trends box draws; the first cut wrote there and would have blanked those trends, the same lesson as snap.flow vs snap.queueFlow); flow.json stays what it is (7 days of rows, the strip's trend table reads it) and is not the history. The history write is its own step with its own log line (history ok only after the object is read back; history error otherwise) and never fails the flow pass: by then the GitHub budget is spent and flow.json is published, and a retry would spend it again.

2. flow: one headline row per reading, and an explicit gap for every silence. Rows are the flow report's headline keys (the same ones flow.json keeps), unioned with the rows already in flow.json so the readings taken before this existed seed it. A row within half a cadence (3 h) of another is the same reading and the newer one wins, so a forced rerun never doubles a point. Wherever two consecutive readings are more than 9 h apart (1.5 cadences), a {no_reading: true, from, to} row is written between them. That is what a resumed collector says about the time it was down. The other half of the property lives in the page: a day with no record renders as "no reading", never as zero, because a collector that is still down writes nothing at all.

3. main: one record per UTC day for main's full test, from post-merge-heavy.yml event=schedule runs (the :07/:37 clock; a hand dispatch is not the clock). A red spell opens at the START of the first failed run (that code was already broken when the run began) and closes at the COMPLETION of the next green run (main is only known green then), in completion order. cancelled is no verdict: it neither opens nor closes a spell. Red hours are clipped to the day; a spell counts, with its time-to-green, on the day it opened; today's record is partial with through; a day with no completed run is no_reading. Each pass fetches the last 2 days plus today (2 REST calls, inside the budget-checked pass) and rewrites those three records; older ones stand. Positive control over the 108 real runs of 1–3 Oct: 5 / 3 / 2 spells and a 1.87 h median time-to-green, the proposal's 5 / 3 / 2 and 1.9 h; red hours 13.1 / 7.5 / 2.1 against its 11.6 / 6.0 / 1.5, the difference being this start-to-completion clip, now written down.

4. reds: facts about each real red, no verdict. From the report's per-run array that flow.json drops: one event per culprit for real_failure and inherited_from_main (its first red run; cascade = the rework runs attributed to it, reds = its own red runs), one per infra run (no culprit). ejectedAt is the queue removal ci-factory-mq-health has CONFIRMED against the PR timeline (its health-cache.json, same box, no API cost), else null = unknown — never false, because the health pass confirms at most 8 per tick and most are simply unchecked. Kept 72 h. What "handled" means (ejected, cascade ≤ 2, green within target) is the page's judgment, so the target can move without rewriting history.

5. Backfill is the same instrument, stamped. The queue side cannot be backfilled from health-cache.json: it holds per-run conclusions, not cascade sizes, queue dwell or first-entry rates, and an approximation under the same keys would be a different measure wearing the same name. Instead the flow script was run by hand over the UTC days 1 and 2 Oct (from the dedicated runtime worktree, the real cache, capped) and the 1 Oct report imported with --import-report --main-days 3, its row stamped source: backfill. The 2 Oct report was deliberately not imported: its window ends 2026-10-03T00:00Z, 35 minutes from the live reading ending 00:35Z, inside the half-cadence "same reading" rule, so it would have displaced a live row; 2 Oct's daily point is covered by that 00:35Z reading (a reading is filed under the day its window mostly covers, until − 12 h). The 1 Oct report's own landed control was weak (139 of 227 queue-merged PRs matched; the queue had started at 22:40Z the night before), which is why the row carries its stamp. Main's side backfills fully from the heavy runs. What could not be produced within budget is left as no reading and said so.

Why this shape

• Keep, do not re-measure. Everything was already computed every 6 hours and thrown away. No new GitHub load beyond 2 REST calls a pass; the one-time backfill was two capped hand runs.
• No new system. A second collector would be a second lock, a second budget check and a second log to watch. The history is a step of the pass that has the report in memory; its liveness is one more history ok signal on the existing ci-factory-mq entry.
• Gaps are data. Both mechanisms (the written gap row, the page's "no record = no reading") are needed; either alone lets a dead collector read as a clean day.
• Targets live in the page. The history records what happened; the proposal's targets are reviewed on 17 Oct and may move. A judgment baked into the store would have to be rewritten then.

Relation to other ADRs

• ADR-0009: the gap row and the history error line are the failure signals; the SYSTEMS.md signal reads history ok with the same 13 h window as the flow signal.
• ADR-0003: the page learns of a dead collector from the record's shape, not by calling anything.
• ADR-0004: this changes how the collector and the docs page relate (a new object between them), so it is written here at decision time.

Build

Known limits at close: --import-report runs under the pass lock but has no budget gate of its own (the hand backfill checked the budget first); the scorecard's XmR mean and limits, once 10 days exist, include today's partial point; the forced pass at 23:41Z moved the next scheduled pass to about 05:41Z.