From sample to world: the central limit theorem

In the previous lesson we learned: an estimate (say, a mean) is a random variable with its own sampling distribution. Now we learn the SHAPE of that distribution for the mean — a surprisingly simple fact on which all further inference rests.

Samples collected: 0n: 10Standard error σ/√n ≈ 3.25x̄ deviation from truth: 0.4

Sample size n10

Before you are three panels. Top — the population: skewed, e.g. service response time (many fast requests, rare very slow ones). Middle — one random sample from it: its points and the green line, the sample mean x̄. Bottom — the distribution of those means, accumulating. Press "take a sample" and notice: x̄ almost never equals the true mean — it misses slightly (the deviation is shown on the right).

you can flip beats with ← → keys

What it means

The main consequence of the CLT feels like a miracle: a tiny part lets you judge the whole. A single sample does lie, of course — by chance it catches somewhat richer or somewhat poorer people, and its mean will not match the truth. But it lies predictably: the means of different samples scatter around the true value in a neat bell, not at random. And once we know the shape of that scatter, we know how far our one sample could have missed. That is why polling 1,500 people reliably gauges the mood of a whole country: what matters is not the share of people covered, but that a random sample of sufficient size misses by a predictably small amount.

How small is set by the standard error σ/√n, and it hides an annoying detail: precision grows as the square root of sample size, not in proportion to it. Halving the error takes four times the data. The first few hundred observations cut the uncertainty sharply, after which each additional thousand adds less and less — which is why polls stop at around 1,500: the error is already about 2–3%, and going further is expensive and nearly pointless.

Where it shows up

Quality control checks the mean of 30 parts, not the whole batch. Ratings and indexes average samples. All of it works because means behave predictably.

A/B tests compare the means of two user groups — and lean on exactly this fact about the behavior of means.

Definitions

Population

all the objects we want to conclude about (e.g. all users).

Sample

a randomly selected part of the population that we actually observe.

Central limit theorem (CLT)

the distribution of the means of random samples tends to normal (a bell) — even if the data itself is not normal.

Standard errorσ / √n

the width of the distribution of sample means: how far a sample mean typically strays from the truth. It shrinks as 1/√n: halving it takes four times the data.

When the method lies (assumptions)

The CLT needs independent observations and finite variance. With ultra-heavy tails (a power law with α ≤ 2) the variance is infinite — no bell forms, and the mean does not stabilize even for large n.

With strong skew and small n the mean's distribution has not yet turned normal — you need a bigger sample. Dependent or clustered data (one user contributing many observations) effectively shrinks the real n.

Deep dive: the math and the mechanism (optional)

Why a bell, of all shapes? The mean is a sum of n independent contributions divided by n. Every observation pulls the mean up or down; when you add them up those random deviations partly cancel each other out, and extreme combinations (every observation large at once) are astronomically rare — so the result piles up symmetrically around the center. The surprising part is that the shape of the original data barely matters: add up many independent contributions and you always arrive at the normal shape.

Where the √n comes from: variances of independent quantities add up (from the previous lesson), so Var(sum) = n·σ², and dividing by n shrinks the variance by a factor of n² → Var(mean) = σ²/n, and σ(mean) = σ/√n. The square root is a direct consequence of the fact that it is variances (squares) that add, not the spreads themselves.