What is Backtesting? How to Test a Strategy Honestly

Why It Matters

Backtesting is how a trading idea earns the right to touch real money. Instead of trusting a strategy because it looks logical, you replay it against historical data and count what actually happens: win rate, average risk-reward, worst losing streak, maximum drawdown.

Done honestly, backtesting answers the question that saves accounts: does this edge exist at all? It also produces the numbers a professional risk plan is built from — you cannot set a sensible risk per trade without knowing your strategy’s realistic losing streaks.

Done dishonestly, backtesting is the most dangerous tool in trading. A test can be tuned until history looks perfect — a failure mode called overfitting — and modern AI tools make generating such “perfect” strategies effortless. The result is a beautiful equity curve that memorized the past and collapses on contact with the future. This failure mode is central to why most AI-generated bots fail, covered in detail in Can AI Actually Improve Your Trading?.

Visual Explanation

In-Sample vs Out-of-Sample

IN-SAMPLE (TUNED)

OUT-OF-SAMPLE (HONEST)

OVERFITTED

ROBUST EDGE

A strategy tuned on one period of history (in-sample) performs perfectly there, but the honest verdict comes from untouched data (out-of-sample), where overfitted strategies collapse

Real Trading Example

A trader builds a breakout strategy for NAS100 and backtests it on 2023 data, adjusting the entry filter until the year shows a 68% win rate. Impressed, they fund an account — and lose steadily for six weeks.

The fix is procedural, not clever. They re-test the original, untuned rules on 2021, 2022 and 2024 data the strategy has never seen, subtract realistic spread and slippage from every trade, and the win rate lands at 52% with a 1.6R average winner — a modest but real edge. Forward-testing it for a month at minimum risk confirms the live numbers match. That smaller, honest edge — sized correctly — is the one that passes an evaluation; the 68% fantasy was the one that ended the first account.

Common Mistakes

Common Mistake

Optimizing until the past looks perfect. Every parameter you tune to improve a historical result makes the test less predictive of the future. If you adjusted the rules after seeing the results, the results no longer count — you must re-validate on data the new rules have never touched.

Professional Tips

Pro Tip

Backtest for the losing streaks, not the profits. The most valuable number in your results is the worst run of consecutive losses. That number sets your survivable risk per trade under a prop firm’s drawdown rules — and tells you in advance that a five-loss week is normal variance, not a broken strategy.

FAQ

How much data is enough for a backtest?

Enough to cover different market conditions — trending, ranging, and volatile periods — and to produce a meaningful sample of trades. For an intraday strategy, aim for several hundred trades across at least one to two years of data. Twenty trades from one good month tells you nothing.

What is the difference between backtesting and forward testing?

Backtesting replays a strategy on historical data; forward testing runs it in real time on a demo account or at minimal live size. Forward testing is slower but exposes everything backtests hide: spreads, slippage, execution delays, and your own psychology. A strategy earns live capital only after surviving both.

Can I trust a backtest generated by AI?

Treat it as a hypothesis, never as evidence. AI tools make it trivially easy to overfit — they can search thousands of parameter combinations until history looks perfect. The validation burden is unchanged: out-of-sample testing, realistic costs, and a forward test. An AI-generated strategy that hasn’t survived those steps is a chart, not an edge.

Backtesting

30-Second Definition