An automated search in our lab found a model with a 90% win rate, a profit factor of 5.3 and a 0% maximum drawdown. It wasn't a strategy: it was a bug in how we decided who won when the stop and the target fell inside the same bar.
What it looked like
A machine learning model for Nasdaq futures (NQ) on 1-minute bars, with a 3-point target and stop: 806 trades, almost $26,000 in simulated profit, a 90% win rate and not a single losing day. Too good to be true. And it was.
The bug
To train and score a model you have to decide, for each bar, whether price hit the target or the stop first. When a bar touches both, the bar doesn't tell you which came first. The labeler assumed "stop first", which is the cautious choice for a buy. But the model traded sells, and the scoring counted those ties as wins. Every ambiguous bar was a free winning trade.
How many were there? Plenty: a 1-minute NQ bar moved about 9.05 points on average, more than the 6 points between the stop and the target.
The check
We rebuilt the labels with 34 million ticks: a tick has a single price, so there is no tie to invent. 9.9% of the labels changed, all in the same direction: from "the sell wins" to "the sell loses".
With equal stop and target, a coin flip is what you'd expect, and that's what came out. The model never exceeded 68.4% confidence, so the search thresholds were unreachable. Of 56 combinations, only one still "passed", with 8 trades: noise. And with its costs (0.80 points per trade), that system needed a 63.3% win rate just to break even.
And the 0% drawdown
The filter meant to protect us had three holes: it measured drawdown on daily results (if no day closed red, drawdown was zero, even if a lot was lost intraday), it didn't include the starting balance in the peak, and one of its criteria was hard-wired to "pass". None of them raised an error.
It wasn't the only time
Months later, the same kind of bug showed up when testing another AI, this time with order flow: its labeler looked at the bar's high first, and 45.7% of the labels were ambiguous. For months we believed the model got 72% of its buys right because the market was rising. It wasn't.
How not to fall for it
- If your target and stop fit inside one bar, resolve the order with ticks or always assume the worst case for the trade you're making, buy or sell.
- Be suspicious of a win rate above 70% with a target equal to or larger than the stop, and of any 0% maximum drawdown.
- Measure drawdown trade by trade, with the starting balance included.
- When something looks spectacular, look for the bug before celebrating. It's almost always there.
Simulated and hypothetical results, with the limitations that implies. Not investment advice. Firm rules change: check them on each firm's website. This article contains no affiliate links.