# Research: CI/test best practices vs PropFlow (2026-09-30)
Labels: SRC = fetched from the URL; LOCAL = from prior PropFlow audits in scratchpad (FACT/EST as labeled there). Not fetched (no claim made): Shopify/Netflix blogs, Bazel/Nx docs, Kent Beck, Accelerate book.

## Principles
| # | Principle | Source | How it applies to PropFlow |
|---|---|---|---|
| 1 | Small, fast, deterministic tests dominate; ~80% unit / 15% integration / 5% e2e (SRC) | https://abseil.io/resources/swe-book/html/ch11.html | Audit shows 38% of runs are no-op "skipped" and ~10 satellite workflows per push; push cost ~55-62 machine-min (EST). Keep pre-merge to fast tests. |
| 2 | Flakiness above ~1% makes tests lose value; Google sits ~0.15% and still sees thousands of flakes/day (SRC) | same SWE-book chapter | 21% of main CI runs cancelled/failed (LOCAL). Measure per-test flake rate; quarantine, do not retry blindly. |
| 3 | Quarantine flaky tests, rerun only known-flaky, run new tests repeatedly before they join the critical path (SRC) | https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html | New tests (mostly agent-written) get a "soak" lane before gating. |
| 4 | Test behaviors, one behavior per test, no loops/conditionals; brittle over-specified tests resist refactoring (SRC) | SWE-book ch. 11 | Rule for agent-written tests: one behavior, assert outcomes not call sequences. |
| 5 | Avoid duplicate coverage across layers; if a high-level test catches what no low-level test did, add a low-level test and delete redundant ones (SRC) | https://martinfowler.com/articles/practical-test-pyramid.html | Prune rule: a test that never fails alone and overlaps another layer is deleted. |
| 6 | Order pipeline by speed, not test type; fast first (SRC) | Fowler, same | Pre-merge = lint/types/affected unit; slow suites move to merge queue or post-merge. |
| 7 | Mutation testing measures whether tests catch injected faults; Google runs it diff-based inside code review, skipping "arid" lines, reaching ~6,000 engineers / ~30% of diffs (SRC) | https://research.google/pubs/state-of-mutation-testing-at-google/ | Run diff-scoped mutation on changed lines only, as a value score for new tests; do not run whole-repo. |
| 8 | Verify before merge (human + machine); re-run CI after push only as a safety net; prefer auto-revert over frequent breakage; keep trunk always green (SRC) | https://trunkbaseddevelopment.com/continuous-integration/ | Matches "Revert button" practice. Post-merge full run should rarely fail; if it does, revert first. |
| 9 | Merge queue validates each PR against latest base, FIFO, never broken by incompatible changes; needs `merge_group` trigger; build concurrency 1-100; min/max group size + wait time for batching; queue jumping forces rebuilds (SRC) | https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue | ~100+ merges/day (LOCAL EST 4,000-4,500/mo) makes stale-base collisions likely. Queue runs the heavy suite once per batch, not per push. |
| 10 | Uber SubmitQueue: speculative builds + conflict analysis keep mainline green; iOS mainline went from green 52% of time to always green (SRC) | https://dl.acm.org/doi/pdf/10.1145/3302424.3303970 | Evidence FOR queues. Our scale (100/day) is far below Uber's; GitHub's native queue with small batches is enough. No evidence fetched AGAINST batching beyond: a bad batch costs bisect/rebuild; keep batch small (2-5). |
| 11 | DORA 2024 elite: lead time < 1 day, on-demand deploys, ~5% change failure, recovery < 1 hour; elite = 19% of respondents (SRC, secondary) | https://octopus.com/blog/2024-devops-performance-clusters | Median PR open->merged 0.65 h, merged->prod 1.13 h (LOCAL EST): lead time already elite. Failure rate and recovery not measured: measure them. Dora.dev page has no numeric tiers (checked). |
| 12 | AI adoption correlates with -7.2% delivery stability and -1.5% throughput; 39% low trust in AI code (SRC) | https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report | Speed is not our problem; stability gates matter more than more tests. |
| 13 | AI assistants raise duplication (5+ line duplicate blocks up 8x in 2024) and cut refactoring (moved lines ~25% -> <10%) (SRC, vendor study, treat as indicative) | https://www.gitclear.com/ai_assistant_code_quality_2025_research | Agents add tests/code and rarely consolidate. Require delete-on-refactor and a test-to-code growth watch. |

## Answers to the five questions
1. Test value: score each test by (a) times it failed on a real defect, (b) mutation kill on changed lines, (c) flake rate, (d) runtime. Prune tests with zero catches in 90 days, high overlap, or flaky. (Practice synthesized from principles 2-7; 90 days is my estimate, not a sourced number.)
2. Gates: pre-merge = lint, types, affected unit tests (minutes). Merge queue = full integration suite once per small batch. Post-merge = slow e2e/evals, auto-revert on red. (Principles 6, 8, 9.)
3. DORA elite: see #11. PR-to-prod shape: one workflow, one deploy per merge batch, revert as recovery path.
4. Agent tests: one behavior per test, outcome assertions, soak new tests before gating, delete with the code they test, track test lines vs code lines. Guardrail thresholds are not sourced; pick from our own baseline.
5. Queues: strong evidence for correctness (Uber, GitHub docs). Batching saves CI at volume; cost is bisecting failed batches. No benchmark found for our size, so pilot and measure.

## PropFlow numbers reused (LOCAL)
- 210,500 sampled runs: 53% success, 38% skipped, 6% cancelled (17% of wall-min), 2% failure.
- Runs per push 32 -> 15 after folding checks (-53%); ~10 satellite workflows still fire per push.
- Median open->merged 0.65 h; merged->prod ~1.13 h (EST).
- PRs >1,000 lines: 9 pushes, 7 of 13 blocking findings; <=300 lines: 0 of 29.