Gating merges on behavior, not vibes

Continuous integration has been able to block a merge on a failing test for twenty years. It has never been able to block one on an answer getting worse. That gap is where most AI regressions live.

One line in your workflow

Add omic run --ci to the workflow you already have. It runs the suite against the pull request branch, compares it to the baseline on main, and exits non-zero when the score drops below your threshold. Your existing branch protection does the rest.

The failing cases are posted on the pull request, with the diff inline. Nobody has to open a dashboard to find out what broke.

Choosing a threshold

Start loose. A gate that fires on every run gets disabled within a week. We suggest setting --fail-under just below your current score, then tightening it as the suite stabilizes.

Slow suites

Split them. Keep a fast suite gating every pull request and run the expensive one nightly. Both write the same file format, so the comparison story does not change.