Walk-forward validation for intraday strategies

Intraday research has an appealing property and a trap concealed inside it. The appealing property is volume: a year of one-minute bars on a single instrument is roughly a hundred thousand observations. The trap is that those observations are not independent, and the effective sample size relevant to any statistical claim is smaller by one or two orders of magnitude.

Walk-forward validation is the correct structure for testing under those conditions. It is also routinely implemented in ways that reintroduce the problem it exists to prevent.

Count what is independent, not what is stored

The relevant number is not observations. It is independent episodes of the phenomenon being studied.

A strategy holding for thirty minutes across a six and a half hour session has at most thirteen non-overlapping opportunities per day, not three hundred and ninety. Across a year that is roughly three thousand, before accounting for the fact that consecutive days share regime, that intraday returns are strongly autocorrelated in volatility, and that a substantial share of the variance concentrates in the open and the close.

Effective sample size shrinks further whenever the signal is a slow-moving state. A feature built from a twenty-day realised volatility estimate changes little from day to day, so observations conditioned on it are close to duplicates. Two thousand rows in which the conditioning variable took four distinct values are four observations wearing a disguise.

Getting this wrong is what makes intraday results look far more significant than they are. A t-statistic computed on overlapping observations is inflated by roughly the square root of the overlap factor, which is large enough to turn noise into a publishable-looking number.

Anchored or rolling, and why it matters

Two structures are in common use and they answer different questions.

Anchored walk-forward keeps the training window's start fixed and extends its end, retraining on all history to date. It suits strategies whose underlying relationship is believed stable, and it uses data efficiently. Its weakness is that the model becomes progressively less responsive as the window grows, and a structural break early in the sample continues to influence estimates indefinitely.

Rolling walk-forward keeps the window a fixed length and slides it. It adapts to regime change and discards distant history. Its weakness is that window length becomes a hyperparameter with real influence, and choosing it by looking at out-of-sample performance converts the out-of-sample set into a training set.

Neither is correct in general. What is not defensible is trying both and reporting the better one, which is a specification search whose cost is invisible in the final number.

Purging and embargoing, done properly

If a label is built from a forward window, training samples whose labels extend into the test period must be removed. Removing them is purging. It is not optional: without it the training set contains outcome information from the test period and the split has failed at its only job.

Embargoing then drops a buffer of observations immediately after the test period before training resumes, because serial correlation means samples adjacent to the test window carry information about it even when their label windows do not formally overlap.

Both gaps need to scale with the actual dependence in the data rather than being set to a round number. The label horizon is a lower bound for the purge. The embargo should reflect the decay of autocorrelation in the features, which is measurable rather than a matter of taste.

The intraday complication is boundaries. A test fold ending at the close and a training fold resuming at the next open are separated by a gap that is short in observations and long in information: overnight news, the auction, and a fresh regime. Splitting at session boundaries rather than at arbitrary row counts is almost always the right choice, and splitting mid-session almost always is not.

The whole procedure is the estimate

The most common error in walk-forward is treating each fold as an independent trial and reporting statistics across folds as though they were a sample.

They are not independent. Folds share hyperparameters, share the feature construction, and in the anchored case share most of their training data. Aggregating them into a mean and a standard error produces a confidence interval that is far too narrow, because the observations underlying it are correlated by construction.

The honest framing is that the entire walk-forward run produces one estimate of one strategy's out-of-sample performance. Uncertainty on that estimate comes from block bootstrap over the concatenated out-of-sample returns, with a block length that reflects the dependence in the series, rather than from the dispersion across folds.

This matters because the second framing tends to produce a comfortable-looking interval and the first frequently does not, and the difference is entirely an artefact of the method.

Where the procedure quietly becomes in-sample

Walk-forward provides no protection at all against decisions made outside it. Every choice made while looking at out-of-sample results is a use of that data.

Selecting the feature set after seeing which folds performed. Adjusting the window length because early results disappointed. Excluding a period as anomalous. Changing the label definition. Each is a legitimate research action and each spends the out-of-sample set, which cannot be regenerated.

The practical control is to fix the specification in writing before the first run, count every variant tried, and hold back a final period that is examined once. If a result only appears after the twelfth variant, the correct interpretation is that twelve specifications were tested, and the multiple-testing correction that implies is severe.

The relationship between this and the curvature argument on the firm's method page is direct. A strategy whose performance is sharply peaked in its parameters has a narrow region of validity and a high probability that the peak is noise. One whose surface is broad and flat around the chosen values is claiming something weaker and more likely to survive. Reporting the surface is more informative than reporting the maximum, and it is the maximum that almost every backtest reports.

A workable default

• Split on session boundaries. Never mid-session.
• Purge by at least the label horizon; embargo by the measured autocorrelation decay.
• Choose anchored or rolling for a stated reason and do not switch based on results.
• Report the concatenated out-of-sample series, not per-fold averages.
• Get uncertainty from a block bootstrap with a justified block length.
• Report the parameter surface alongside the chosen point.
• Count and disclose the number of specifications tried.

The first three cost implementation effort. The last four cost something more uncomfortable, which is that they usually make the result look weaker. That is the point of them.