It’s better to be lucky than good.

Do you regularly read stories about people who failed? Of course you don’t, you prefer to read about those who succeeded. The problem is that, if we spend our life studying successful people and companies that shook the planet, our knowledge of the world will be strongly biased and enormously incomplete.

Bullet holes¹

During World War II the English sent daily bombing raids into Germany. Many planes never returned; those that did were often riddled with bullet holes from anti-air machine guns and German fighters.

Wanting to improve the odds of getting a crew home alive, English engineers studied the locations of the bullet holes. Where the planes were hit most, they reasoned, is where they should attach heavy armor plating. Sure enough, a pattern emerged: Bullets clustered on the wings, tail, and rear gunner’s station. Few bullets were found in the main cockpit or fuel tanks.

They concluded that they should add armor plating to the spots that got hit most often by bullets. Was this correct? This is a stunning example of “survivor bias” — drawing conclusions only from data that is available or convenient and thus systematically biasing our results. Let me explain… Planes with bullets in the cockpit or fuel tanks didn’t make it home; the bullet holes in returning planes were “found” in places that were by definition relatively benign. The real data is in the planes that were shot down, not the ones that survived.

So you want to be the next Steve Jobs?²

The survivor bias was also evident in Walter Isaacson’s 2011 best-selling biography of Steve Jobs, as readers scrambled to understand what made the mercurial genius so successful.  Want to be the next Steve Jobs and create the next Apple? Drop out of college and start a business with your buddies in the garage of your parents’ home. David Cowan of Bessemer Venture Partners writes: “For garage-dwelling entrepreneurs to be successful, their path almost always involves raising venture capital and then getting their start-up to an initial public offering (IPO) or a large acquisition by another company. But VCs hear 200 pitches for every one they fund. So for every successful start-up founder, there are 199 other entrepreneurs who end up with only a cluttered garage.”

No one writes books about them and their unsuccessful companies!

Google, Victoria’s Secret and the role of luck

 Now, consider the story of Larry Page and Sergey Brin.³ Larry and Sergey were two very creative students in the computer science department at Stanford University who came up with a superior way of searching information on the Internet. They sought and obtained funding to start a company and made a series of decisions that worked out well. Within a few years, the company they started, Google, was one of the most valuable stocks in America, and the two former graduate students were among the richest people on the planet.

What is hidden from this success story is the fact that a year after founding Google, they were willing to sell their company for less than $1 million. They eventually didn’t sell the company because the buyer said the price was too high. Mentioning this single lucky incident still underestimates the multitude of ways in which luck affected the outcome.

”When looking for advice, we should look for what not to do, for what is missing. However, we should not expect to find it among the quotes and biographical records of people whose signals rose above the noise.”

Another Stanford graduate observed that it was a universal certainty that most men would rather be in a war zone than in a women’s underwear store. What if there was a nice place that men could feel comfortable in; a shop where they could browse at their leisure without having to manically flash their wedding bands? It led Roy Raymond to found Victoria’s Secret back in 1977.⁴

In marketing only to men, Raymond forgot the basic principle that most of a women’s underwear drawer will be purchased by her and not her other half. Alienating the main consumers of women’s underwear, i.e. women, was probably not the most sensible idea, and in 1982, due to financial stress, Raymond was forced to sell the company to sportswear magnate Leslie Wexner for around $1million.

By 1995, when the brand launched its now iconic catwalk shows, Victoria’s Secret had become a $1.9 billion company, with 670 stores across the US. Today the brand controls a huge 35% of America’s lingerie market (according to Forbes), with sales over $6.6 billion in 2013.

Sadly, despite his original foresight, Raymond did not share in this success. After staying on as president for a year, he left to form another retail and catalogue company, this time in children’s clothes. His brand My Child’s Destiny was declared bankrupt within two years, leaving Raymond personally liable for its debts.

In August of 1993, Roy Raymond jumped to his death from San Francisco’s Golden Gate Bridge. No one wrote a book about Roy Raymond!

Those who fail rarely get paid for advice on how not to fail.

When looking for advice, we should look for what not to do, for what is missing. However, we should not expect to find it among the quotes and biographical records of people whose signals rose above the noise. They may have no idea how or if they lucked up.

Also, keep in mind that those who fail rarely get paid for advice on how not to fail, which is too bad because despite how it may seem, success boils down to serially avoiding catastrophic failure while routinely absorbing manageable damage.

But don’t tell a successful person that he has been lucky. Nobody accepts randomness in his own success, only his failure.

Burying Evidence: The File-Drawer Effect

Survivorship bias has also become a problem in science. Scientific journals only present the survivors of the journal process – studies showing statistical significance. Psychologists call this the File Drawer Effect. The studies that disprove or weaken the hypotheses of high-profile studies seem to get stuffed in the file drawer, so to speak.

Daniele Fanelli sums it up nicely:⁵ “One of the most worrying distortions that scientific knowledge might endure is the loss of negative data. Results that do not confirm expectations—because they yield an effect that is either not statistically significant or just contradicts an hypothesis—are crucial to scientific progress, because this latter is only made possible by a collective self-correcting process. Yet, a lack of null and negative results has been noticed in innumerable fields. Their absence from the literature not only inflates effect size estimates in meta-analyses, thus exaggerating the importance of phenomena, but can also cause a waste of resources replicating research that has already failed, and might even create fields based on completely non-existent phenomena.”

Conclusion

Survivorship bias is our tendency to focus on survivors instead of whatever we would call a non-survivor depending on the situation. Sometimes that means we tend to focus on the living instead of the dead, or on winners instead of losers, or on successes instead of failures.

After any process that leaves behind survivors, the non-survivors are often destroyed or removed from our view. If failures becomes invisible, then naturally you will pay more attention to successes. Not only do we fail to recognize that what is missing might have held important information, we fail to recognize that there is missing information at all.

• Survivorship Bias, You Are Not So Smart, David McRaney, May 23, 2013
• How the Survivorship bias Distorts Reality, Scientific American, Michael Shermer, September 1, 2014
• Thinking Fast and Slow, Daniel Kahneman, Farrar, Straus and Giroux; April 2, 2013
• The tragic story of the man who founded Victoria’s Secret, Elle Blakeman, Telegraph, 2 December 2014
• Negative results are disappearing from most disciplines and countries, Danielle Fanelli, Scientometrics, March 2012, Volume 90, issue 3, pp 891–904