Avoiding Backtesting Biases & Simulation Flaws
Historical simulations can be highly deceptive. Learn how to identify and eliminate look-ahead, survivorship, and selection biases.
01. The Danger of Look-Ahead Bias
Look-ahead bias occurs when future data is used to make decisions in the past. In manual testing, this happens when you look at the right side of the chart before deciding where to enter on the left side.
In automated coding, it occurs when algorithms access variables (like tomorrow's closing price) to calculate today's indicators. To avoid this, always use candle-by-candle step progression, keeping the future completely hidden.
02. Survivorship Bias in Stock Testing
Survivorship bias is the logical error of focusing only on successful assets while ignoring those that failed. In stock backtesting, this occurs when you run your strategy on the current components of the S&P 500.
This ignores the hundreds of companies that went bankrupt, merged, or were delisted over the last 10 years, skewing your results to look highly profitable. To fix this, always include delisted equities data in your historical backtesting datasets.
03. Selection Bias and Selection Filters
Selection bias happens when you select a specific historical window or asset to fit your strategy rules. For example, backtesting a breakout strategy only on technology stocks during the 2020 bull market.
While the strategy will show astronomical profits, it is statistically invalid. A robust system must be tested across different market cycles (bull, bear, range) and across uncorrelated assets (forex, commodities, indices) to verify its true expectancy.