Multiple regression and controlling for variables
Simple regression looks at the relationship of x and y alone — and is therefore easily fooled. The analyst's real workhorse is multiple regression: it estimates the effect of x while holding other factors constant. This is where the practical answer to "correlation ≠ causation" lives.
Set the true effect to 0 and crank up the confounder strength: the pooled red line will show a "relationship" that isn’t there. Controlling for z (within-group lines) removes it. That is what multiple regression does — estimates the effect of x while holding z constant.
Before you are two groups of points by a hidden factor z (gray — z=0, teal — z=1). There is no control right now: the red line is the pooled regression of y on x. It goes up — as if x strongly affected y.
Multiple regression is the analyst's most frequent tool after descriptive statistics. "Does tenure affect salary once role and city are accounted for?", "the effect of a promo on revenue controlling for season and channel" — these are all coefficients, all else equal.
Its main strength and main danger are one and the same: what exactly you put in the model. Forgot an important confounder — the effect is distorted; included the wrong thing (see the collider in the next lesson) — also distorted. So the set of controls is chosen by the logic of the problem, not "everything in".
Econometrics and the social sciences rest almost entirely on multiple regression: estimate the effect of education on income holding age, gender, region constant.
In product work — estimate a feature's contribution to retention controlling for account age and platform, when a clean A/B on that feature is impossible.
Definitions
The "all else equal" coefficient is honest only if the right confounders are in the model and the relationship is roughly linear. An omitted important factor biases the estimate (omitted variable bias).
Strongly correlated predictors (multicollinearity) make individual coefficients unstable. And controlling does not turn observational data into an experiment — the risk of unaccounted factors remains.