What 400 lines of harness taught us

Every AI team we know has one: a homegrown eval harness, a few hundred lines of Python, written in an afternoon and resented ever since. Ours was 400 lines. Omic exists because of what it got wrong.

The four failures

It stored results as printouts. It compared runs by hand. It broke every time the schema moved. And nobody trusted it enough to block a merge on it.

Each failure became a design rule: results are files, comparison is built in, the format is versioned, and the gate is the point.

What survived

One thing the script got right: it ran in one command, with no server. That constraint stayed. Everything else was rebuilt around it.