Not on the Micro Nasdaq. We tried 526 combinations of six textbook strategy families (2019-2026) and none passed a test fixed before looking. The search's best one (t = 2.21) is no better than what the same search finds in a shuffled market with nothing to find (1.21 to 3.32 in 20 tries). And re-optimizing every quarter would have made about −$8.
This is what happens when you test a lot of ideas at once, the way strategy "miners" do. Each test you add raises the bar, and what looks good in the search is mostly luck. We did it with the lab's engine, which fills orders like NinjaTrader, and with real costs: $0.62 per contract per side plus 1 tick of slippage on entry and exit, trading 1 MNQ.
What we tried
5-minute bars of the Micro Nasdaq, split into three periods: search (2019-06 to 2023-10), validation (to 2025-04) and a final test opened only once (to 2026-10).
| Family | Combinations | Best search t | Median validation t | Median test t |
|---|---|---|---|---|
| Opening range breakout | 192 | 2.21 | 0.78 | −0.24 |
| Moving average crossover | 54 | 0.70 | 0.07 | −0.62 |
| RSI reversal | 54 | 0.61 | −1.15 | −0.53 |
| Previous day's high and low | 108 | 1.45 | −0.28 | −0.49 |
| VWAP deviation | 64 | 2.15 | −0.90 | −0.39 |
| Opening gap | 54 | 1.83 | 0.04 | −0.73 |
t: average result per trade divided by its error. Above 2 starts to be unusual for one test; among hundreds, it isn't.
The test
- Criteria sealed before looking (with a date and a fingerprint). Changing them after seeing the result voids the test.
- A bar that rises with the number of tests: with 526 independent tries, chance alone gets a t of about 3.54. None got there. Then neighbors, validation, double costs and the final test: nothing passed the first step.
The control: the same search in a market with nothing to find
We shuffled the real 1-minute bars across days, each one with another from the same New York minute of the day. The shuffled market keeps each hour's volatility, the fat tails and the overnight gaps, but one bar says nothing about the next: there's no edge to find. Then we ran the exact same mining 20 times, each on a different shuffle.
| Real market | Shuffled market (20 tries) | |
|---|---|---|
| Best search t | 2.21 | 1.21 to 3.32 |
| Search-validation correlation | 0.40 | 0.03 to 0.40 |
| Combinations that pass the test | 0 | 0 in all 20 |
| Walk-forward (23 quarters) | −$8 | $28,778 to −$13,744 |
| Average combination | −$887 | −$151 to −$7,205 |
What it means
- The best one in the search isn't better than chance. Its t of 2.21 is inside what the shuffled market gives. Its validation t was −0.09 and its test t −0.50.
- "The search predicts validation" proves nothing either. A correlation of 0.40 also shows up in the shuffled market (up to 0.40), probably because a trend that lasts for years favors the same families in every period.
- The walk-forward doesn't tell anything apart either. The real one ends at −$8, and 7 of the 20 shuffles did as well or better. One made $28,778 (t = 1.96) in a market with nothing to find: a good walk-forward can be luck too.
- The only hint: the average combination loses less in the real market than in 19 of the 20 shuffles (by luck that would happen about 1 time in 10). Something may be there, but it doesn't make money after costs.
- The theoretical bar is conservative. 3.54 assumes independent tests; similar combinations behave like fewer tests. In the 20 shuffles, the best reached at most 3.32, and 19 times in 20 it stayed under 2.88. Comparing with the shuffled market is the fair bar. Even so, the real best (2.21) didn't clear it.
What about the opening range breakout from the 7-year study?
It fits. In that study the 30-minute breakout was examined on its own: one idea, built on recent data and checked on the earlier years. Here, the long-only version with the same 30-minute range comes out positive in all three periods (t of 1.22, 0.93 and 0.76), but with a t around 1 it can't stand out among 526. Studying one idea with a hypothesis isn't the same as searching through hundreds and keeping the best.
What to take away
- Count every test. Each parameter, filter and time window you tried counts, not just the version you kept.
- Demand data you didn't use and compare with chance before paying for an evaluation. Here is how many trades you need, and what held up two tests when we did it carefully.
Educational content based on our own experience. Not investment advice. Firm rules change: check them on each firm's website. This article contains no affiliate links.