No. 269 · Evidence · Heuristics and Bias
Selection Bias
How you choose a sample can decide the answer before you ask a single question.
In the Almanac
“The Literary Digest poll”, 1936 · “Progressive Snapshot”, 2011
Both stories in full, three ways to use it and where it misleads.
Get the AlmanacQuestions
What is Selection Bias?
Selection Bias occurs when the people or data you study are not representative of the population you want to understand. If the way the sample is chosen favours certain types, conclusions will be skewed, however large the sample.
How do you avoid Selection Bias?
Whenever you see impressive statistics, testimonials or survey results, ask how the group was chosen and who was left out. Look for fair comparisons between similar groups, and seek views beyond your own circle before assuming everyone agrees.
When does Selection Bias matter?
Selection Bias is one of the 68 mental models for evidence, the situations where you ask: Is this really true? Within that, it belongs to “When research goes wrong”, alongside Anecdotal Evidence, Data Dredging and Hasty Generalisation.
What is an example of Selection Bias?
Two stories show it at work: The Literary Digest poll (1936) and Progressive Snapshot (2011). Both are told in full in The Mental Models Daily Almanac, which has two stories for each of its 365 models.
Which mental models are related to Selection Bias?
Survivorship Bias, Adverse Selection and Confounding Factor. Selection Bias itself comes from heuristics and bias.