Interactive statistics
Module 10: Data traps
Progresscompleted 0 / 63

Goodhart's law: the metric as a target

The last trap is about the metrics themselves. As soon as a metric is made a target, people start gaming it, and it stops reflecting what it was introduced for.

Target metricReal valueE.g.: "calls per hour" โ†‘ while conversation quality โ†“
While the pressure is moderate, the metric and the value grow together.

A call center decided to push the "calls per hour" metric. Move the pressure on the metric. At first both the metric and the real value (the quality of the conversation with the customer) grow together โ€” all is well.

What it means

When setting goals (OKRs, KPIs), experienced teams always ask: "how can this goal be gamed to the detriment of the business?" โ€” and add a counter-metric that will catch it.

This is a direct bridge to the second part of the site: in every metric hierarchy the bottom level is guardrails. They exist precisely because of Goodhart's law.

Where it shows up

Schools that drill for the test instead of knowledge; hospitals turning away severe patients to improve mortality statistics; social networks burning out their audience for momentary engagement.

Everywhere the same: one indicator got optimized โ€” and the system it was supposed to measure broke.

Definitions

Goodhart's law
when a measure becomes a target, it ceases to be a good measure โ€” people start gaming it.
Gaming the metric
formally improving the indicator to the detriment of the real goal it was introduced for.
Counter-metrics (guardrails)
quality constraints next to the target metric that catch gaming and degradation.
Next โ†’
Breakdowns like this live in the channel

Data analytics in plain words: how to count metrics, how not to fool yourself in an A/B test, what gets asked in interviews and how this site gets built. The channel is in Russian.

Open the Telegram channel