Overfitting auditor
A backtest is not evidence until you say how many you ran
Search forty-five variants of a strategy over five years of daily data, with no skill whatsoever, and the best one you find will have a Sharpe ratio of about 1.0. That is not a warning about sloppy research — it is arithmetic, and it means a 1.0 Sharpe discovered that way carries no information at all. Paste a curve below and this will tell you where yours sits against the search you say produced it.
This reads your equity curve, not your code. It cannot see look-ahead, survivorship bias in your universe, fills you would never have got, or costs you left out — those live in the backtest, not in its output. It also has to take your trial count on trust. What it can do is tell you how much the number you already have is worth given the search you say produced it.
Paste a column straight out of a spreadsheet. Several columns — one per configuration you compared — unlock the split test. Nothing here is uploaded; the arithmetic runs in this tab.
Every parameter set, indicator swap and threshold you looked at before settling here — including the ones you discarded. There is no default, because a number chosen after the result is known is the problem this measures.
What noise alone will hand you
The first column is how many configurations were tried. The second is the Sharpe ratio the luckiest of them is expected to show over 5 years when none of them has any edge. The third is how much history you would need before a search that size could be told apart from skill at a Sharpe of 1.0.
| Variants tried | Best Sharpe from luck alone | History needed |
|---|---|---|
| 1 | 0.00 | 0.0 yr |
| 5 | 0.53 | 1.4 yr |
| 20 | 0.85 | 3.6 yr |
| 45 | 1.00 | 5.0 yr |
| 100 | 1.13 | 6.4 yr |
| 500 | 1.37 | 9.3 yr |
| 2,000 | 1.54 | 11.9 yr |
Computed at render by the same functions the auditor above runs on, over 5 years of daily data. The independence assumption behind them is conservative: real searches are correlated, which makes the true no-skill maximum lower than these figures.
Where the numbers come from
The Deflated Sharpe Ratio and the no-skill maximum are Bailey and López de Prado (2014); the minimum backtest length is Bailey, Borwein, López de Prado and Zhu (2014); the track-record length is Bailey and López de Prado (2012); the split test is the Combinatorially Symmetric Cross-Validation of Bailey, Borwein, López de Prado and Zhu (2015); the serial-correlation correction is Lo (2002), and the reason it matters for smoothed returns is Getmansky, Lo and Makarov (2004).
None of this is ours. What is ours is that it runs on your numbers, for free. We have not put our own Momentum Score through this arithmetic — what we publish about it instead is that it has never been tested against forward returns at all, see how the score is computed.
What this cannot see
It reads an equity curve, which is an output. Most of the ways a backtest misleads are properties of the code that produced it: look-ahead, where a signal quietly used tomorrow's close; survivorship bias, where the universe excludes the companies that went bankrupt; fills at prices nobody could have got; costs left out. None of those are visible here, and a clean result on this page is not a clean bill of health on any of them.
It also cannot check the number you typed. The whole calculation turns on how many variants you actually tried, and that figure is on trust — which is why the result says so, every time, rather than presenting a number you supplied as one we measured.