During WWII, British bombers were often taken down by the German defense lines.
Experts at the British Air Ministry mapped out where bullets had hit the planes that actually made it back.
The next step?
Add extra armor to the areas with the most holes.
But the extra armor adds weight.
Too much of it, and the planes would fly like refrigerators.
So they needed to reinforce only the critical spots.
Luckily, they decided to ask mathematician Abraham Wald for some insights.
They were stunned with his proposal:
“Nope. Reinforce the places with the fewest bullet holes.”
The heck?
The damage map had been created using the surviving planes (those that could take hits in certain areas and still returned home).
The planes hit in the blank areas had not returned and were therefore not included in the analysis.
That’s the survivorship bias: drawing conclusions only from the data we see, while ignoring the silent evidence of what’s missing.
If we consider the data we don’t see, we would likely get different conclusions.
- We glorify the entrepreneur who bet everything and won, while the stories of all those who followed the same strategy but failed receive less coverage (despite being more common).
- We obsess over the startups launched in garages by 22-year-olds, even though companies founded by older entrepreneurs survive much longer.
- We have all heard stories of centenarians who started smoking in their 20s, without suffering any health issues. But less is said about those who died of lung cancer before reaching old age.
This bias infects science itself.
Studies that observe a positive effect of a drug are more likely to be published than those that show no benefit.
When we check published studies, it seems like every drug is a miracle pill.
But that’s only because we don’t have access to the studies buried in the cemetery of unpublished negative results.
When making a decision, take some time to look beyond the available information.
Ask yourself what information might be missing and what its absence might indicate.
Of course, study the success stories, but also learn from the ones that failed.