CI Factory: one app instead of two pages
The plan to replace /a/ci-factory and /a/ci-factory-floor with one clean app at factory.propflowai.co: what it answers, how it's built, and the order we ship it.
Plan · Oct 6, 2026 · Gera's CI session · nothing built yet · built by a Fable driver once approved
One app, three questions, four screens. Today two pages show about 30 panels, each on its own snapshot and its own time window. The app answers three questions instead, over any time range you pick: Is CI healthy right now? Is it getting better or worse? What should we fix next, and what is it costing us?
- Screens: Now (the floor), Health (trends), Reds (what to fix), Cost. Plus a Log of who tried what (with markers on the charts), and click-through pages for one PR, one test, one run.
- Backend: one database of every CI event (runs, failures, PR timelines, deploys), filled by one collector, queried for any range. It replaces the six snapshot files.
- Hosting: factory.propflowai.co on Cloudflare, behind the same company login as the docs site (Google only, straight to Google's sign-in). Kept separate from the main app, so it still works when CI is broken.
- Order: data first, then Reds with a time-range picker (first visible win), then Health, Now and Cost. The old pages stay until the app covers them.
Detail
Why change: what's wrong today
- Two pages, about 30 panels. The floor page is the top of the full page, plus three strips added this week (flow, scorecard, reds). Nothing tells you where to look first.
- Every panel has its own clock. Reds cover 7 days, the flow strip 24 hours, the scorecard 14 days, the pass-rate chart 72 hours. You can't line them up or change them.
- Six snapshot files, two machines. factory.json comes from a collector on Fede's Mac; mq, mq-health, floor, flow and history come from the mini. Each is rebuilt whole and overwritten, so nothing older than its window can be asked about.
- Numbers that mislead without context. For example "14 failed" counts test runs, not PRs, and most are knock-on from one bad PR. "Merge-queue contexts" is a summary job that looks like a cause.
The four screens (and what's on each)
| Screen | Answers | On it | Comes from today |
| Now (home) | What's happening this minute? | The floor: PRs moving through stations, the queue line, the deploy countdown, an alert bar (main red, the culprit just kicked out, cheap machines being lost) | The floor engine, deploy card, culprit line |
| Health | Better or worse? | The 5 scorecard measures plus opened→live, each with target and trend; the queue pass-rate chart split into "PR's own fault" and "knock-on" | Scorecard, flow strip, pass-rate chart |
| Reds | What should we fix next? | Red causes ranked by cost: still happening vs fixed, times it happened, runs taken down, ≈$, PR-hours; the flaky-test list | "Why it went red", reds chart, flaky table |
| Cost | What does CI cost, and where is the waste? | Machine $ per day, waste from red runs, review-bot spend, AWS breakdown | Spend section, machine bays, cost tile |
On every screen: one time-range picker (24h · 48h · 7d · 30d · all) that applies everywhere; PR search; every number has a hover that says exactly what it counts and links to the raw runs.
Click-through pages: a PR (its trip through each station, every queue run, why it was kicked out, time lost), a test (every failure, own vs knock-on, first and last seen, whether it's fixed), a run (jobs, the failing step, the log link).
Backend: one store of events, not six snapshots
- Database: Cloudflare D1 (SQLite), next to the app. It stores events, never pre-computed totals, so any range or split is a query: runs (PR, queue, main, release), failed jobs with step, test and cause class (own / knock-on / machine / flaky), PR timelines (opened, approved, queued, kicked out, merged, live), deploys, machine cost per run.
- Ingest: one collector on the mini, reading GitHub as the propflow-agent-smith App (its own request budget). It's woken by the existing review webhook on each queue or run event, with a 15-minute catch-up sweep. It reuses today's logic, which classifies each red's cause and confirms the culprit from the PR timeline, instead of rewriting it.
- Backfill: everything since Sept 30 from the mini's run cache, plus GitHub's deploy records. Older history only as far as GitHub keeps it.
- Fede's collector (machine bays, AWS spend) pushes into the same store, or the app reads its file during the transition. To agree with CICD.
- Honesty rules, built in: a missed collection is stored as "no reading", never zero. Every number keeps a link to the runs it came from.
Frontend
- React + Vite on Cloudflare Pages at factory.propflowai.co, behind Cloudflare Access with Google as the only sign-in, set to go straight to Google (no Cloudflare page). Light and dark, works on a phone.
- One chart library, the house palette. Each screen leads with one headline number and its trend; detail is one click away, never on the first screen.
- Plain English labels. Knock-on and machine failures are always shown apart from real ones.
Order of work (each step ships on its own)
- Data: D1 schema, the collector writing events, the backfill. Check: totals match today's pages for the same 24 h.
- Reds screen + time-range picker, with the own vs knock-on split. First visible win, about 1–2 days in.
- Health screen: scorecard and trends over any range.
- Now screen: the floor moves over and becomes the home page.
- Cost screen + click-through pages.
- Retire the old pages: both URLs point to the app.
Each step is small PRs off main, reviewed, measured. Claimed in the engineering bridge first so CICD reviews instead of building in parallel.
Design: how it looks and stays calm
Five rules that keep it from overwhelming:
- Every screen opens with one sentence, the verdict. For example, "Queue 78% green today, up from 68% yesterday. Biggest cost: staff-desk test, 6 PRs kicked out, ≈$4, 3 PR-hours." Read that and stop, or scroll for why.
- At most six numbers above the fold. Each one shows its target and its change since the previous period (▲ 4 pts). Everything else is one click away, in a side drawer rather than another panel.
- One clock. The time range in the top bar drives every chart and table, and each is compared with the period before it.
- Noise is grey and folded by default. Knock-on failures and lost machines appear as grey or striped segments, apart from real red. "Probably fixed" causes are collapsed.
- Every number explains itself. A hover says what it counts, and a click shows the exact runs or PRs behind it.
Navigation: a slim left rail (bottom tabs on a phone): Now · Health · Reds · Cost. Top bar: time range (24h · 48h · 7d · 30d · All · custom), PR search (type #17039), a live status pill (main green/red · queue depth · next deploy in 12 min), theme.
Charts, one kind per job:
- Pass rate over time: stacked bars per hour or day. Green = passed, red = the PR's own fault, amber = knock-on, grey = machine lost. A dashed target line at 95%.
- Each measure (Health): a tile with a sparkline. Click it for a large line chart with the target band, the normal-variation limits once there are 10+ days, and "no reading" gaps.
- What to fix (Reds): a ranked bar list (a Pareto) showing which few causes make up 80% of the waste. Click a cause to open a drawer with its timeline strip, affected PRs, cost and fix status.
- Money (Cost): stacked area of $ per day: useful testing, knock-on waste, machine-loss waste, review bot.
- One PR: a horizontal journey bar (opened → checks → review rounds → queued → kicked out → re-queued → merged → live), with each wait labelled in minutes.
- Now: today's floor (stations with PR dots), the queue line, the deploy countdown, plus a live event feed on the right (merged, kicked out + why, deployed, main red).
┌ Factory ───────────────────────────────── [24h|48h|7d|30d|All] #____ ● main green · queue 9 · deploy 12m ┐
│ Now │ HEALTH Queue 78% green today, ▲10 pts. Opened→live 2.6 h, ▼0.2 h. 3 of 6 targets met. │
│ Health ◀│ ┌Queue green┐┌Main green┐┌Fix main┐┌Wasted/PR┐┌Merged 1st┐┌Opened→live┐ │
│ Reds │ │ 78% ▲10 ││ 100% ││ — ││ 4 runs ││ 81% ▲6 ││ 2.6h ▼0.2 │ ← target ✓/✗ each │
│ Cost │ └~~~~~~~~~~~┘└~~~~~~~~~~┘└~~~~~~~~┘└~~~~~~~~~┘└~~~~~~~~~~┘└~~~~~~~~~~~┘ │
│ │ Pass rate by hour ▇▇▅▇▇▇▃▇▇ green=pass red=own amber=knock-on grey=machine - - 95% target │
│ ⚙ Targets│ [click a tile → large chart drawer · click a bar → the runs in that hour] │
└─────────┴───────────────────────────────────────────────────────────────────────────────────────────────┘
Settings: kept small. Targets per measure, each change dated so history stays honest. Default range. Timezone (Central). Optional Slack alerts (main red > N min, queue green < X% for an hour).
As much data as is useful: every table sorts and filters and exports as CSV, and every chart point drills down to its rows. Depth lives behind clicks, never on the first screen.
Effort log: who tried what, and did it work
Gera: "right now we are kind of battling… I don't know who's doing what." The app gets a fifth screen, Log, plus markers on every chart.
- One line per effort: when, who (session, lane or person, e.g. "Gera · ci session" or "Fede · CICD"), what ("moved queue CI Checks to spot + in-run retry"), the PR link, and the measure it's meant to move ("queue green", "merge→live").
- How lines get in: agents write one with a tiny command (
factory-log "…" --pr N --aims queue-green), and the collector adds one automatically for every merged CI PR and every red-main claim or resolution in the bridge. Humans can add one from the app.
- Markers on charts: each effort appears as a small flag on the trend charts at the moment it landed, so you can see whether the line moved after it. Click a flag for the one-liner and its PR.
- After 48 h, the app writes the result next to the line: "queue green 68% → 79% in the 48 h after", or "no change". Efforts that did nothing are visible, not forgotten.
Keep the feel, rebuild from scratch
- Kept as ideas, rebuilt fresh: the floor's moving dots between stations, the red/green trend bars, the recent reds, the scorecard. Less text: labels become hovers, paragraphs become one-line verdicts.
- Look and feel to borrow from: Vercel's and Linear's dashboards (calm, lots of white space, one accent colour), Trunk.io and Buildkite's CI and flaky-test views (cause → tests → runs drill-down), and GitHub's merge-queue timeline (the PR journey bar). The driver studies those, then designs ours. No copying of code or branding.
- Motion with a purpose: dots travel between stations when a PR moves; a red pulse on the station where something just failed; nothing animates for decoration. Respects reduced-motion.
Decisions (made Oct 6)
- Name: factory.propflowai.co.
- Hosting: its own pipeline on Cloudflare, not inside the main app.
- Login: Cloudflare Access with Google only, straight to Google's sign-in. A custom sign-in page is a possible later polish.
- Fede's spend data: pulled into the same store (agreed with CICD during the build).
Done means: you can answer the three questions in under a minute, for any range, and the old pages are retired.