Pick an option on each card and press Done; your answer goes straight
back to the session. The raiser's pick is marked. "I'm not sure" is a real answer.
1
In plain terms. Should the daily Clara behavior check on PRs only report NEW failures, and should someone own fixing main's eval runs that have been failing since 9/27? It doesn't block merges, but it marks prompt PRs failed for problems main already has, and tonight sessions held PRs they thought were blocked.
The full question, as raised
Corrects b91421597. clara-behavior-evals feeds PR Verdict, but PR Verdict isn't a required check (branch protection requires Build, Type Check, Unit Tests, Regression Gate — Leasing, Lint, review; queue runs Nested Gate on merge_group). #17618 and #17623 merged at 00:46Z/00:48Z with PR Verdict FAILURE. Main's 5 promptfoo domain jobs in Nightly Stress Tests failed 09-27/28/29 and 10-04 (run 37194490990), no owner. Option A: grade PR sweep against main's latest sweep, report only green-on-main/red-on-PR, run full sweep on main post-merge or daily with a named owner per red gate on the nightly audit.
Why, and what was tried
The session raising this recommends: PRs only fail for new problems; main's get owners Stops PRs looking broken for main's old problems, and gets main's red checks fixed by someone named. Tried first, unsuccessfully: Trinity traced it; Smith confirmed the nightly reds, that PR Verdict isn't required, and that #17618/#17623 merged with it red.
c1e3a896 is parked on this.
Answered and withdrawn (1)
b91421597 Should the daily Clara behavior check on PRs only block on NEW failures, instead of failing PRs for problems already broken on main? Right now main's own weekly run has been red since at least 9/27, so whichever prompt PR draws the one daily check gets blocked for things it didn't break. Four PRs hit this after midnight today. — folded into b91421673