Set up CI
Running evals locally tells you what changed. Running them in CI stops a regression from ever reaching main — it’s the first thing most teams do once their first suite works.
Omic runs the same command in CI that you run locally, so there’s no second configuration to keep in sync.
Add it to your pipeline
One step, in whatever CI you already use:
•
omic run --ci — runs the suite and emits machine-readable output
•
--baseline main — compares this branch against your main branch
•
--fail-under 0.9 — exits non-zero when the score drops below a threshold
Gating a merge
A failing suite behaves exactly like a failing test:
•
Non-zero exit — blocks the merge through your existing branch protection.
•
PR comment — the failing cases and their diffs are posted inline.
•
No new infrastructure — it’s a CLI call; nothing needs hosting.
Keeping history
Local runs are ephemeral. Connect Cloud when you want them to persist:
•
Shared run history — every run from every branch, in one place.
•
Cross-commit comparison — see when a metric started sliding, and why.
•
Shared links — send a run instead of a screenshot.