Interactive statistics
Module 2: Probability
Progresscompleted 0 / 63

Conditional probability and rare events

Randomness does not only deceive on small samples. There is a trickier trap: even a very accurate test for a rare disease produces a flood of false positives. Let's see why โ€” this is conditional probability.

Out of 1000 people, โ‰ˆ 10 are sick; the test is positive for โ‰ˆ 108.
All positive tests:actually sick: 9false positives: 99
P(sick | test positive) = 8%
Bayesโ€™ theorem (live numbers):
P(sick | +) = P(sick)ยทP(+|sick) / [ P(sick)ยทP(+|sick) + P(healthy)ยทP(+|healthy) ]
= (0.010ยท0.90) / [ 0.010ยท0.90 + 0.990ยท0.10 ] = 8%
P(sick) is the base rate (prevalence), P(+|sick) is the test accuracy. Drag the sliders โ€” the formula recomputes.

Take a disease that affects 1 person in 100, and a test that is 90% accurate. The test comes back positive for some people. The bar shows everyone whose test fired: teal โ€” actually sick, yellow โ€” healthy people with a false positive.

Why this matters

This trap is real in medicine: when screening for rare diseases, most positive results turn out false, so the first test is followed by a confirmatory, more precise one.

The same effect drives fraud and spam filters: if fraud is rare, even a good detector will bury you in false positives unless the base rate is taken into account.

Where it shows up

Breathalyzers, lie detectors, "suspicious person" recognition in crowds โ€” all suffer from this: with a rare event and good accuracy, the absolute number of false positives is huge.

A competent conclusion therefore always weighs not only "how accurate is the test" but also "how rare is the event itself".

Definitions

Conditional probability
the probability of an event given that another is known. P(sick | test +) is not the same as P(test + | sick).
Base rate (prior)
how common the event is in general, before any test. Rarity drastically changes what a positive result means.
Bayesโ€™ theoremP(AโˆฃB)=P(A)โ‹…P(BโˆฃA)P(B)P(A \mid B) = \dfrac{P(A)\cdot P(B \mid A)}{P(B)}
how to update the probability of A after observing B, accounting for both the test's accuracy and the base rate P(A).
Next โ†’
Breakdowns like this live in the channel

Data analytics in plain words: how to count metrics, how not to fool yourself in an A/B test, what gets asked in interviews and how this site gets built. The channel is in Russian.

Open the Telegram channel