No. 1 · Decisions · Systems
A/B Testing
When opinions clash, change one thing and let real behaviour decide.
In the Almanac
“Hopkins’s coded coupons”, 1923 · “Optimizely”, 2008–2010
Both stories in full, three ways to use it and where it misleads.
Get the AlmanacListen · Watch
Questions
What is A/B Testing?
A/B Testing means showing two versions to similar groups and letting real behaviour pick the winner. Claude Hopkins used coded coupons to test ads in the early 1900s, and Dan Siroker's tests for the Obama campaign raised sign-ups about 40 percent before he co-founded Optimizely.
How do you use A/B Testing?
Try it with email subject lines, budgeting methods or daily routines. Change one thing at a time, measure what happens, and give it enough time to matter. Just don't let small tests distract from bigger questions.
When does A/B Testing matter?
A/B Testing is one of the 35 mental models for decisions, the situations where you ask: Which way should I go? Within that, it belongs to “Weighing the options”, alongside Back-of-the-Envelope Calculation, Cost-Benefit Analysis and Decision Tree.
What is an example of A/B Testing?
Two stories show it at work: Hopkins’s coded coupons (1923) and Optimizely (2008–2010). Both are told in full in The Mental Models Daily Almanac, which has two stories for each of its 365 models.
Which mental models are related to A/B Testing?
Randomised Controlled Experiments, Hypothesis and Minimum Viable Product. A/B Testing itself comes from systems.