Trusting the Harness
Read the verdict like a pro
The Harness is the lie detector. Before a single dollar goes live, it replays history walk-forward — train on the past, test on a future it never saw — and asks one question: did this model actually beat the closing market out of sample? It hands back a verdict. Here's how to read it.
The four verdicts
Out-of-sample pari-mutuel ROI is positive with the confidence interval clearing zero. This is the only verdict that unlocks live betting. Real, paid, repeatable.
Positive signal but not yet significant — or strong CLV with flat ROI (sharp, but only cashable at fixed odds). Keep it on paper and gather more races.
It does not beat the close. No threshold or tweak fixes this — the features need work. Do not bet.
Too few settled bets to judge. The honest answer is "we don't know yet." Get more data.
The numbers behind the verdict
The bottom line — profit per $1 if paid at final odds. This is what makes EDGE real. Watch the CI: the lower bound must clear 0.
Leading indicator. Are our bets beating the close? Goes positive before ROI does. Trust it earliest.
Can the model rank winners as well as the market? 0.5 = coin flip. The closing market is ~0.77 — that's the wall.
When we say 20%, do they win 20%? Low ECE = honest probabilities. High = overconfident (the classic failure).
Why CLV can be great while ROI is flat
You'll see it: CLV +12%, ROI −2%. That's not a contradiction. It means your reads are genuinely sharp (you beat the close) but in the pari-mutuel pool you're paid the close, so the edge doesn't fully cash. The fix is when you bet (late), not whether the model works.