The bootstrap: an interval without formulas
We built the confidence interval leaning on formulas and normality. But what if the data is odd and no suitable formula is at hand? There is an ingeniously simple trick โ the bootstrap: resample the sample from itself.
On top is our only sample of 15 values (right-skewed). The bootstrap works in steps: 1) draw the same number of values from it randomly WITH REPLACEMENT (one value may appear several times, another not at all), 2) compute their mean, 3) drop that mean into the bottom distribution. Press "1 resample (step)" โ green points of the drawn values pour in on top, their green mean appears, and via the diagonal link it "falls" into the bottom histogram.
What it means
At first the bootstrap feels like cheating: how can you learn about an estimate's reliability without collecting anything new, just by reshuffling the same data? The secret is that the sample itself is our best (and only) map of the population. Draw values from it at random with replacement and you get a "pseudo-sample" โ the kind you might have collected on a parallel attempt. Compute the metric on a thousand such pseudo-samples and you see with your own eyes how much it varies from sample to sample โ the very uncertainty formulas used to be derived for.
The bootstrap is the workhorse when formulas are missing or data is non-normal: a confidence interval for median revenue, for conversion at small counts, for any custom dashboard metric.
The same principle underlies many machine-learning methods (bagging, random forests): train models on resampled datasets and average.
Where it shows up
An analyst needs to know how reliable the median call duration is โ the formulas for the median are unwieldy, while the bootstrap gives an interval in a couple of lines of code.
In science the bootstrap saves the day when the distribution is unknown: instead of dubious assumptions, the data "tells its own story" about its variability.
The name comes from "pulling yourself up by your bootstraps" (Baron Munchausen pulled himself by the hair): Bradley Efron proposed exactly that in 1979 โ the sample pulls the estimate of its own uncertainty out of itself, with no new data.
Definitions
When the method lies (assumptions)
The bootstrap recomposes what is already in the sample: if 30 observations failed to catch the distribution's tail, it will not appear in 10,000 pseudo-samples either. On small n the intervals come out too narrow โ optimistic.
The classic bootstrap requires independent observations: time series and clusters need special variants (block, cluster bootstrap). And for extreme statistics โ the maximum, the minimum, rare quantiles โ it is unreliable in principle.