EXP-000001
Equity-basis vs balance-basis drawdown across the catalogue
Hypothesis
Balance-basis drawdown (closed trades only) materially understates peak-to-trough risk for systems that hold losing positions, and must not be ranked against equity-basis figures silently.
Controlled change
Importer reads equity drawdown wherever the report states it and records the basis; every page carrying a drawdown states which basis it shows; balance-basis pages carry an explicit caveat.
Dataset & conditions
All 14 tester reports (5 MT5 with both bases, 9 MT4 balance-only).
Before / After
Before
- SUPREMACY drawdown as first imported (balance basis)
- 12.44%
After
- SUPREMACY drawdown on equity basis, same report
- 48.82%
Validation
Out-of-sample, walk-forward, Monte Carlo and forward results are shown only where they exist as data. None exist for this entry.
Decision
The gap on the worst case was a factor of ~4 (12.44% vs 48.82%). Publishing the balance figure as "the" drawdown would have materially understated risk.
AI involvement
- Model
- Claude (Anthropic)
- What AI did
- Read all 14 tester reports, surfaced the equity/balance mismatch and drafted the importer change. The publication rule (always label the basis) was decided by a human.
- Human review
- ✓ Reviewed and approved by a human
AI-assisted entries are published only after human review; entries generated by AI without that review are withheld by the build pipeline. AI does not predict markets, and no entry claims otherwise.
Notes
Basis labelling now appears on detail pages, the compare matrix and the risk/return map.
Backtest improvement does not imply future improvement. The more experiments run, the more likely some succeed by chance — this log exists partly so that multiple-testing risk stays visible instead of hidden.