Correlation: Pearson, Spearman and what r misses
The correlation coefficient r is a handy single number about a relationship. But it measures only ONE kind of relationship — the linear one — and it is easy to fool. An experienced analyst never trusts r without looking at the chart itself. Switch between the point clouds and compare Pearson with Spearman.
Try the four "Anscombe" presets in a row — Pearson r stays ≈0.82 in every one, even though the shapes are nothing alike.
Pearson sums the "rectangles" (x−x̄)(y−ȳ): blue ones (both coordinates on the same side of the mean) pull r up, red ones pull it down; the sum is normalized by the spread. Spearman does the same but first replaces the values with their RANKS (places) — so it catches any monotonic relationship and is robust to outliers: on "An outlier hides the relationship", Pearson collapses to zero while Spearman on ranks stays high.
Straight to the two charts and the formula. On the left — the cloud by values with the cross of means (x̄, ȳ); this is Pearson: r = Σ(x−x̄)(y−ȳ) / √(Σ(x−x̄)²·Σ(y−ȳ)²). Each point contributes (x−x̄)(y−ȳ): blue points (both coordinates on the same side of the mean) pull r up, red ones pull it down; the sum is divided by the spread so r lands in [−1, 1]. On the right — the same points, but by RANKS (places): Spearman's ρ is the same Pearson computed on ranks. On "Linear" the points stretch in a straight band, almost all blue — Pearson is high and deservedly so.
Anscombe's quartet, which we just switched through in the widget, isn't just a textbook legend: the same r ≈ 0.82 genuinely sits behind four completely different pictures. One moral: a correlation number without a chart is deceptive.
In practice, for ratings, placements, scores (ordinal data) and for relationships with outliers you take Spearman. For clean linear measurements — Pearson. But you always look at the scatter.
"Salary vs tenure" is often monotonic but not linear (growth slows) — Pearson will understate the relationship, Spearman shows it more honestly.
In monitoring, a single anomalous day easily paints or breaks a correlation between metrics — so relationships are re-checked without outliers and with rank methods.
Definitions
Pearson reflects a relationship correctly only if it is roughly linear and there are no strong outliers; otherwise it systematically misleads.
Any correlation (Pearson or Spearman) is a relationship, not causation. A high coefficient does not mean one thing causes the other.