Custom scorers
When no built-in scorer fits, write your own. A scorer is a function that takes an output and returns a number — that is the whole contract.
Writing one
Register a function and use it by name:
•
@omic.scorer — the decorator that registers your function
•
returns 0.0 to 1.0 — the score for a single case
•
Any Python — call a library, a service, or your own model.
Using it in a suite
Reference it exactly like a built-in:
•
scorers = [my_scorer] — declared in omic.toml
•
Per case — override the scorer for a single case when you need to.
•
Combined — several scorers can run against the same output.
Keeping them trustworthy
A scorer is only useful if it is stable:
•
Deterministic — the same output must always produce the same score.
•
Fast — it runs once per case, on every pull request.
•
Versioned — changing a scorer invalidates your baseline — bump it deliberately.