Network effects: when A/B breaks
The pizzeria tests a "bring a friend — you both get a discount" promo on a random half of its users. But friends are shared: the test group invites the control group, and the "clean" comparison falls apart. A regular A/B test rests on a quiet assumption: my landing in group B does not affect a user in group A. In social networks, marketplaces and delivery it collapses — and the naive group comparison gives a biased answer.
User split: connected people land in different groups, B’s effect "leaks" to their friends in the control (red links), the control rises — and the naive estimate of B’s effect is understated. Cluster randomization (whole communities in one arm) removes almost all cross-group links and gives a clean estimate. A practical way to DETECT interference: estimate the effect both ways — a large gap between them is what gives away the SUTVA violation.
The hidden assumption of A/B is called SUTVA: each user's outcome depends only on THEIR variant, not on what others got. User-level randomization is honest only when people don't interact. Often that is true — and then regular A/B works.
If users influence each other, a regular A/B will give a biased answer. The design decision — clusters or switchback — is made before launch.
Before an A/B on a platform where users interact, think about the unit of randomization right away: people, clusters, geo or time. Getting this wrong means a pretty but wrong result that falls apart at rollout.
The telltale of interference is shared resources or communication between users. A marketplace with a shared courier pool almost always calls for a geo or switchback experiment, not a user split.
Social networks test viral mechanics by clusters (by community) — otherwise the effect "leaks" through friendship ties into the control.
Delivery and ride-hailing services use switchback and geo experiments: if the test takes couriers away from the control, a regular user split will show a false result.
Definitions
Cluster and geo designs are more honest under interference, but costlier: the effective sample size is the number of clusters, not people, so you need many clusters for power.
Switchback assumes the effect shows up quickly and there is no strong carryover between periods; with long delayed effects it is biased too.