OpenAI
Omic calls providers through a thin adapter, so the suite you write does not change when the model does. OpenAI is the most common provider and works with no extra configuration.
Configuring
Point a suite at a model:
•
OPENAI_API_KEY — read from the environment
•
model = gpt-4o — set per suite in omic.toml
•
temperature — pinned per run so scores stay comparable
Model versions
Version pinning is what keeps runs comparable:
•
Pin explicitly — an alias that moves silently invalidates your baseline.
•
Recorded per run — the exact model is stored in the .eno file.
•
Flagged on change — Omic warns when a diff spans two different models.
Cost and rate limits
Evals call models a lot, so:
•
Responses are cached — an unchanged case is not re-run, or re-billed.
•
Concurrency is capped — tune it to match your rate limit.
•
Retries are automatic — transient failures do not fail the suite.