Neglecting the invisible
Routinely throughout history, a single and apparently improbable decision has saved thousands of lives.
In World War II, while vying for air supremacy in the Pacific, American generals confronted an increasingly serious problem of their bombers being taken out by anti-aircraft guns. The generals were under enormous pressure to identify the vulnerabilities in their air fleet, and to immediately recommend a mitigation plan.
Analysts inspected all the bombers that had returned to home base, frantically searching for clues. Soon enough they noticed a pattern in the damage these planes had sustained: bullet holes were concentrated in the tail and wings.
The conclusion seemed obvious: reinforce these damaged areas of the planes to increase their odds of survival.
But a young statistician gingerly raised his hand to voice an objection. "Our analysis is based only on the data from planes that have returned. What about those that never made it back? What brought them down?"
With that insightful question, the generals soon realized that the observable data was concealing what truly mattered. They decided to reinforce the undamaged areas of returning planes, narrowly escaping yet more tragedy.
Our tendency to neglect the invisible also shows up when we overlook the good: diseases that have been prevented, conflicts that have been averted, downturns that have been forestalled. Such things are hard to imagine, let alone measure.
It might be true that a famous person's career strategy was the key element behind his success. It might also be true that the same strategy failed for many others we've never heard of.
It might be true that a software firm is losing revenue because of issues that have been raised in customer engagement sessions. It might also be true that the majority of renewals are churning for reasons that are never discussed, because other customers have lost interest in continuing a relationship.
It might be true that a top performer is focusing on the wrong things because she's behind on the metrics you manage. It might also be true that she's doing exactly what she should be, and that the scope of your metrics prevents you from noticing it.
And so on.
Most of our thinking about what's working and what's not is based on that which is right in front of us. Too often we miss the obvious question: "What data am I missing?"