No. 326 · Fixing problems · Systems
Black Box
When you cannot see how a system reaches its answers, you cannot easily spot its mistakes.
In the Almanac
“Warren’s flight recorder”, 1954–1967 · “COMPAS”, 2016
Both stories in full, three ways to use it and where it misleads.
Get the AlmanacListen · Watch
Questions
What is Black Box?
A Black Box is any system we can use but can't see inside. David Warren invented the cockpit flight recorder in 1958 to reveal what happened in crashes. Decades later, ProPublica reported the COMPAS score wrongly flagged Black defendants nearly twice as often; its maker disputed it.
How do you use Black Box?
When stakes are high, ask what factors a system uses, test it against cases you already understand, and be wary of products nobody can explain. And open your own black box by explaining your reasoning to others.
When does Black Box matter?
Black Box is one of the 40 mental models for fixing problems, the situations where you ask: Why does this keep going wrong, and how do I fix it for good? Within that, it belongs to “Finding the real cause”, alongside First Principles Thinking, Inversion and Lateral Thinking.
What is an example of Black Box?
Two stories show it at work: Warren’s flight recorder (1954–1967) and COMPAS (2016). Both are told in full in The Mental Models Daily Almanac, which has two stories for each of its 365 models.
Which mental models are related to Black Box?
The Map is not the Territory, Garbage In Garbage Out and Post Mortem. Black Box itself comes from systems.