Overfitting: how good-looking numbers lie

What overfitting is

A strategy is overfit when its rules describe the noise in one stretch of history instead of a repeatable behavior of the market. Here is the seductive part: the rules really did work on that history. The backtest is not lying about the past. It is silent about the future, and you heard a promise in the silence.

The test is old and simple. Fit on one period. Run on a period the rules never saw. If the performance collapses, the “edge” was a description of dead noise. Most strategies collapse. That is not pessimism. That is the base rate.

Every parameter you add is another degree of freedom, which is another way to memorize history instead of learning from it. More knobs, more memory, more risk that memory is all you have.

How it hides

Overfitting never announces itself. It arrives dressed as hard work. Each of these feels like diligence while you do it.

• Indicator stacking. RSI below 30, and MACD crossing, and price above the 200 MA, and volume elevated. Each condition trims trades. Stack enough and you have written a precise description of one historical window. The window will not happen again.
• Parameter shopping. Run 5,000 backtests, report the best. The best of 5,000 random strategies is always brilliant. That is a certainty of arithmetic, not evidence of edge.
• Asset shopping. Twenty pairs, keep the green one. Same contest, different costume. See single-pair backtests.
• Timeframe shopping. Six timeframes, keep the green one. Same disease, different organ.
• Look-ahead bias. Using information the strategy could not have had at trade time. The next bar's close. Revised data. This is not even overfitting. It is time travel, and it inflates results just as reliably.

The warning signs

Treat each of these as a presumption of guilt until the strategy proves otherwise:

• Sharpe above 3 on anything that is not explicit market-making.
• Win rate above 70 percent with a reasonable reward-to-risk. Check that against the trader's equation and ask what is being hidden.
• A tiny in-sample drawdown. Markets do not give smooth rides. Optimizers do.
• Performance that collapses when a parameter moves 1 percent. A real edge is a region. A fitted curve is a spike.
• Performance that collapses on the neighboring pair or timeframe.

One sign is a question. Several together are an answer.

The defenses

Every defense is the same idea wearing different clothes: confront the strategy with data it was not fitted to.

• Out-of-sample testing. Hold data back. Judge there. The strongest single defense. See out-of-sample testing.
• Walk forward. Re-fit on rolling windows, score on what follows each. Shows whether the edge persists or happened once.
• Stability checks. Nudge every parameter. Real edges survive nudges.
• Multi-objective optimization. Optimizing return, drawdown, and frequency together is harder to game than one number. See Pareto frontiers.
• Cross-asset, cross-regime. Does the edge exist on BTC and ETH and SOL? In the bull year and the chop year? Edges that exist once, in one place, usually never existed.

Calibrate your eye

What real results look like, after costs:

• Sharpe 1.0 to 1.5 with out-of-sample confirmation: probably real and genuinely good.
• Sharpe 2.0 plus, one asset, no holdout: probably fit.
• Sharpe 3.0 plus: almost certainly fit, unless there is an explicit microstructure reason.

The summary nobody enjoys: if the backtest looks too good to be true, the probability is that it is not true. The strategies that survive live trading are the ones whose backtests looked modestly good. Modest is what real looks like.

How Edgecraft handles this

Out-of-sample validation and cross-pair resilience and parameter robustness tests are built into every optimization study. It is not a step you can skip when the numbers get exciting, because exciting numbers are exactly when traders skip it. Parameter sets that die under small perturbations get flagged as fragile instead of crowned. Pareto frontiers keep one gamed metric from hiding the damage to the others. And if an overfit strategy slips through anyway, live strategy health analysis compares live results to the backtest envelope and announces the divergence in weeks. The market would also announce it. The market charges more for the news.