The bootstrap: an interval without formulas

We built the confidence interval leaning on formulas and normality. But what if the data is odd and no suitable formula is at hand? There is an ingeniously simple trick — the bootstrap: resample the sample from itself.

Resamples: 1

● resample values / latest mean▮ accumulated means▮ 95% bounds

The algorithm: 1) draw n values from the sample with replacement (green dots above) · 2) compute their mean (the green dot) · 3) it drops into the bottom distribution · repeat N times.

On top is our only sample of 15 values (right-skewed). The bootstrap works in steps: 1) draw the same number of values from it randomly WITH REPLACEMENT (one value may appear several times, another not at all), 2) compute their mean, 3) drop that mean into the bottom distribution. Press "1 resample (step)" — green points of the drawn values pour in on top, their green mean appears, and via the diagonal link it "falls" into the bottom histogram.

you can flip beats with ← → keys

What it means

The bootstrap is the workhorse when formulas are missing or data is non-normal: a confidence interval for median revenue, for conversion at small counts, for any custom dashboard metric.

The same principle underlies many machine-learning methods (bagging, random forests): train models on resampled datasets and average.

Where it shows up

An analyst needs to know how reliable the median call duration is — the formulas for the median are unwieldy, while the bootstrap gives an interval in a couple of lines of code.

In science the bootstrap saves the day when the distribution is unknown: instead of dubious assumptions, the data "tells its own story" about its variability.

The name comes from "pulling yourself up by your bootstraps" (Baron Munchausen pulled himself by the hair): Bradley Efron proposed exactly that in 1979 — the sample pulls the estimate of its own uncertainty out of itself, with no new data.

Definitions

Bootstrap

estimating uncertainty by repeatedly recomposing the sample from itself.

Resampling with replacement

draw n values from the sample, returning each drawn value — values may repeat.

Percentile interval

a confidence interval as the 2.5th and 97.5th percentiles of the bootstrap distribution.

When the method lies (assumptions)

The bootstrap recomposes what is already in the sample: if 30 observations failed to catch the distribution's tail, it will not appear in 10,000 pseudo-samples either. On small n the intervals come out too narrow — optimistic.

The classic bootstrap requires independent observations: time series and clusters need special variants (block, cluster bootstrap). And for extreme statistics — the maximum, the minimum, rare quantiles — it is unreliable in principle.