Trading · Expert · 13 min lesson
Trading expert: strategy testing, drawdown control and psychology of discipline
Out-of-sample testing, costs and sample size, drawdown arithmetic and limits, and rule-based discipline (DDS-3 sections 1 to 8; ESMA CFD measures 2018). Demo accounts only
What you will learn
- Test a strategy honestly with a written hypothesis, out-of-sample data, walk-forward checks, costs and a log of every variant tried.
- Calculate drawdown and recovery, and set risk per trade, daily loss, weekly loss and size-down limits that are written before trading.
- Build discipline with pre-commitment rules, a checklist and a rule-compliance score instead of relying on willpower.
Key takeaways
- Write the hypothesis first, log every variant tried, and report out-of-sample results beside in-sample results (DDS-3 sections 1, 4, 5).
- Costs and slippage belong in the test: a gross edge of 0.25 R can fall to 0.12 R net.
- Recovery needed equals loss divided by one minus loss, so a 50 percent drawdown needs a 100 percent gain.
- Write limits for trade, day, week and drawdown before trading, and reduce size after losses, never increase it.
- Discipline comes from pre-commitment, a checklist, a journal and a compliance score. Education only; demo accounts; risk of loss.
Free practice question
Dana tested thirty variants of a moving average rule on 24 months of data and reports the best one, plus 64 percent. She reports no out-of-sample result and does not mention the other twenty nine variants.
What is the main concern?
The best of thirty is partly luck, and without the variant count or out-of-sample result the edge cannot be judged.
Correct. DDS-3 section 4 requires the variant count and section 5 the out-of-sample result, so luck can be separated from edge.
Thirty variants is too few to prove anything.
Not quite. Section 4 sets no minimum; the problem is that the number tried and the out-of-sample result are missing.
Moving average rules cannot be tested on demo data.
Not quite. Section 2 allows testing on history and demo data; the concern is selection bias rather than the rule type.
Plus 64 percent is too high to be real.
Not quite. Section 5 does not judge by size; it asks for evidence on unseen data to decide whether the result is real.
Key terms
- in-sample data: The history used to design and tune a rule; results here are biased upward.
- out-of-sample data: History held back and examined once after the rule is frozen, to test whether the edge survives.
- walk-forward test: Design on one window, test on the next period, then roll both forward and repeat.
- overfitting: Fitting a rule to noise in past data so that it looks strong in testing and fails afterwards.
- expectancy: Average result per trade, often stated in R, after wins, losses and costs.
- R multiple: The result of a trade divided by the amount risked on it; one R is the initial risk.
- drawdown: The fall in account equity from a peak to a later low.
- pre-commitment: Rules fixed before trading, so the decision is not made under emotional pressure.
Official sources
- Bailey, Borwein, Lopez de Prado and Zhu, The Probability of Backtest Overfitting (SSRN)
- ESMA adopts final product intervention measures on CFDs and binary options
Go further
The full lesson has narrated slides, more scenario questions with explained answers and a simulation.
Open the AcademyWatch on YouTubeEducational content, not financial advice. Trading involves risk of loss.