Conditional probability and rare events
Randomness does not only deceive on small samples. There is a trickier trap: even a very accurate test for a rare disease produces a flood of false alarms. Let's see why β this is conditional probability.
Take a disease that affects 1 person in 100, and a test that is 90% accurate. The test comes back positive for some people. The bar shows everyone whose test fired: teal β actually sick, yellow β healthy people with a false alarm.
This trap is real in medicine: when screening for rare diseases, most positive results turn out false, so the first test is followed by a confirmatory, more precise one.
The same effect drives fraud and spam filters: if fraud is rare, even a good detector will bury you in false alarms unless the base rate is taken into account.
Breathalyzers, lie detectors, "suspicious person" recognition in crowds β all suffer from this: with a rare event and good accuracy, the absolute number of false alarms is huge.
A competent conclusion therefore always weighs not only "how accurate is the test" but also "how rare is the event itself".