Interactive statistics

56 free interactive lessons: from mean and median to A/B tests and Bayes. Drag the charts and build intuition. No sign-up.

Module 9: Data traps

Simpson's paradox

The next trap hides in the relationships between quantities themselves. Sometimes the trend visible in the data as a whole flips completely once you split it into groups. This is Simpson's paradox β€” it wrecks naive conclusions and regularly catches even seasoned analysts.

Across all the points the relationship is downward (negative).

Before you is a cloud of points β€” say, employee tenure horizontally, salary vertically. Across all the points the line goes down: as if more tenure meant a lower salary. A strange conclusion.

What it means

The classic real example: a drug helped less overall, but better in every patient subgroup (mild and severe cases). The severe cases simply got it more often and dragged the overall result down.

That is why analysts almost reflexively slice data by key segments: device, region, age. The same chart "overall" and "by segment" can tell opposite stories.

Where it shows up

The paradox has been caught in university admissions, hospital comparisons, ad effectiveness by channel. Everywhere the "average over everyone" misled until the data was split by group.

The lesson is simple: aggregates are convenient but dangerous. Always keep in mind which groups they are mixing.

Definitions
Simpson's paradox
a relationship visible in the pooled data disappears or flips sign when split into groups.
Hidden variable
an unaccounted factor (here β€” the department) that creates the false overall trend.
Aggregation
pooling groups into one pile. Convenient, but can distort or flip a relationship.
Next β†’
Enjoying the materials?

This site is built by the DataSlice Telegram channel: data analytics in plain words β€” real cases, metrics, careers. The channel is in Russian.

Open the Telegram channel