Interactive statistics
Module 10: Data traps
Progresscompleted 0 / 63

Survivorship bias

The most insidious kind of bias is when the data you do NOT see matters more than the data you have. The classic example โ€” WWII airplanes.

โ— bullet holes of the returned
The military wanted to reinforce where there were most holes โ€” the wings and the tail. Where is the armor actually needed?

The military studied planes that returned from missions to decide where to add armor. The white dots are bullet holes. Sounds logical to reinforce where there are most of them โ€” the wings and the tail?

What it means

"College dropouts become billionaires" is textbook survivorship bias. For every Zuckerberg there are thousands of dropouts with no success, and they are simply absent from that sample.

The same in product work: you survey the remaining users and hear everything is fine โ€” while the key churn data sits with those who left silently and never made it to the survey.

Where it shows up

"How to succeed" books dissect companies that survived โ€” that is why they get written about. Those that did the same and went broke never made the sample.

Investors see the returns of funds that still exist; the ones closed at a loss quietly vanish from the statistics, and the average return looks better than reality.

Definitions

Survivorship bias
concluding from those who "lived" into the data while ignoring those who dropped out of it.
Selection bias
the more general case: the sample is unrepresentative because not everyone makes it in.
Next โ†’
Breakdowns like this live in the channel

Data analytics in plain words: how to count metrics, how not to fool yourself in an A/B test, what gets asked in interviews and how this site gets built. The channel is in Russian.

Open the Telegram channel