We Rejected 4 of Our Last 5 Models. Here's Why That's the Point

Most betting sites tell you their model “keeps improving.” They don’t tell you what that means in practice: how many candidate versions get tried, how many get thrown away, or what the bar for “good enough to ship” actually is. In July 2026 we ran five model candidates across four sports through our testing pipeline. One passed. Here’s the full record — including the four that didn’t.

The test we run on every model

Before any model version touches a live prediction, it goes through the same gate. We split our history database at a cutoff date: everything before the cutoff is training data, everything after is a reserved holdout window the model never sees during training. Once training finishes, we run the model forward through the holdout exactly as if it were making picks in real time — game by game, using only information that would have been available on that date — and grade every pick against what actually happened.

That distinction matters more than it sounds. A model that trains on data including its own test window can look good for a reason that has nothing to do with skill: it has effectively memorized outcomes it’s being “tested” on. We caught this exact problem in our own history — an earlier NCAAB candidate looked like it had an edge until we noticed its backtest window overlapped its training data. The edge vanished once we fixed the split. Every model discussed below was evaluated on a window that ends after training data stops, with no overlap.

The gates

A model doesn’t get activated for looking good. It has to clear a fixed, sport-specific bar on the holdout window:

• Spread sports (NCAAB, NBA): at least 52.4% against the spread — the break-even rate once you account for standard -110 vig — and positive ROI.
• Moneyline sports (MLB, NHL): at least 54% win rate and positive ROI.

Both conditions have to hold. A model that hits the win-rate number but loses money on the odds it was actually laying (favorites cost more than they pay) fails. A model with positive ROI built on a lucky small sample fails the win-rate check. The gate is deliberately hard to game.

The July 2026 scorecard

One activation. Four rejections. That’s not a bad month for the model — it’s the process working exactly as designed.

NCAAB v3.0.0: the one that cleared the bar

On 2,849 graded games in the holdout window, NCAAB v3.0.0 went 804-686 against the spread — 54.0%, comfortably clear of the 52.4% break-even line — and returned +2.9% ROI. Both gates passed on a sample large enough that the result isn’t noise. This is the version making NCAAB picks on the site right now.

MLB v1.0.0 and v1.1.0: two swings, two misses

Our first MLB moneyline model lost money on the holdout — -10.5% ROI — badly enough that there was no ambiguity about the verdict. We didn’t ship it and move on; we went back and added starting-pitcher form and park-factor features (the kind of inputs our park-factor research found matter for MLB totals) and retrained as v1.1.0. It improved — -5.9% ROI is meaningfully better than -10.5% — but “less negative” is still negative. The gate requires positive ROI, not improvement. v1.1.0 was rejected too. MLB moneyline picks are not live on the site because nothing has cleared the bar yet.

NHL v2.1.0: the case for two gates, not one

NHL v2.1.0 is the most instructive rejection of the batch, because it looks like a pass at first glance: +6.0% ROI, which sounds like exactly what you want. It failed anyway, because it missed the 54% win-rate gate, and when we looked at its calibration — how well its stated confidence matched its actual hit rate — it was noisy: the model wasn’t consistently right when it said it was confident. A positive ROI built on inconsistent calibration is the kind of result that looks like an edge in one sample and a loss in the next. That’s exactly why we require both conditions, not just one. See our calibration guide for what well-calibrated confidence actually looks like, using the NCAAB v3.0.0 numbers as the contrast case.

NBA v2.0.0: a clean, boring rejection

NBA v2.0.0 wasn’t close on either measure — 51.0% ATS is below the 52.4% break-even line, and -2.3% ROI confirms it. No controversy here, no near-miss story. It simply didn’t have an edge on this holdout window, and it doesn’t run in production.

Why we’re telling you this

A one-for-five hit rate on model candidates might read as a knock against us. We think it’s the opposite. Every prediction service claims its model is good; almost none show you the versions that failed the test before you saw the one that passed. If a site never mentions a rejected model, ask yourself whether that’s because they’ve never had one — or because they don’t publish it.

We publish every graded pick from every active model on our results page, win or lose, and we’re now publishing the process that decides which models get to make picks at all. The bar is fixed in advance. Models that don’t clear it don’t go live, no matter how promising they looked during training. That’s the whole point of a holdout window: it’s the one part of the process a model can’t talk its way past.

What this means for you as a bettor

• NCAAB picks on this site come from a model that beat a strict, pre-registered bar on nearly 2,850 unseen games — not a curated highlight reel.
• We don’t currently publish MLB moneyline or NHL picks from a model that has cleared its gate, because none has yet. If you don’t see a market covered here, that’s the reason — not an oversight.
• “Improved” is not the same as “good enough.” MLB v1.1.0 was a real improvement over v1.0.0 and still got rejected, because the bar is positive ROI, not “better than last time.”
• Ask any service you’re trusting with money how they test. If they can’t describe a holdout window that never overlaps training data, be skeptical of any number they show you.