Interactive statistics
Module 10: Data traps
Progresscompleted 0 / 63

Simpson's paradox

The next trap hides in the relationships between quantities themselves. Sometimes the trend visible in the data as a whole flips completely once you split it into groups. This is Simpson's paradox โ€” it wrecks naive conclusions and regularly catches even seasoned analysts.

Across all the points the relationship is downward (negative).

Before you is a cloud of points โ€” say, employee tenure horizontally, salary vertically. Across all the points the line goes down: as if more tenure meant a lower salary. A strange conclusion.

What it means

The classic real example: a drug helped less overall, but better in every patient subgroup (mild and severe cases). The severe cases simply got it more often and dragged the overall result down.

That is why analysts almost reflexively slice data by key segments: device, region, age. The same chart "overall" and "by segment" can tell opposite stories.

Where it shows up

The paradox has been caught in university admissions, hospital comparisons, ad effectiveness by channel. Everywhere the "average over everyone" misled until the data was split by group.

The lesson is simple: aggregates are convenient but dangerous. Always keep in mind which groups they are mixing.

Definitions

Simpson's paradox
a relationship visible in the pooled data disappears or flips sign when split into groups.
Hidden variable
an unaccounted factor (here โ€” the department) that creates the false overall trend.
Aggregation
pooling groups into one pile. Convenient, but can distort or flip a relationship.
Next โ†’
Breakdowns like this live in the channel

Data analytics in plain words: how to count metrics, how not to fool yourself in an A/B test, what gets asked in interviews and how this site gets built. The channel is in Russian.

Open the Telegram channel