Years ago, while attending an avant-garde theater production in a small, artsy space in Boston, my friend suddenly stiffened in her chair and appeared to be struck by a seizure. White fog from the play that was wafting into the seats made me wonder if she was suffering an allergic reaction.
At a loss, I carried her through the theater doors, shouting for a doctor. I was relieved to be followed out by a clean-cut man who took control of the situation and began barking orders to lay her on the ground, raise her head, get some water, etc. As we were doing this, another man arrived. Slightly disheveled, with long hair and hip clothes that made him look like one of the actors, he leaned over my unconscious friend and, watching her the entire time, quietly asked me a series of questions.
Impatient with the questions, the first man stood up and announced, “We’re calling an ambulance, she should get to the ER immediately.”
Still crouching over my friend, the second man said quietly, “There’s no need for that.”
Stiffening, the first man said, “Sir, I am a paramedic.”
“Sir,” the second man said, still quietly, ”I am a doctor.”
Moments later my friend revived, the doctor completed his diagnosis, and all ended well, without a trip to the emergency room.
Recently I thought of the confident paramedic and the curious doctor when I came across a fascinating Money Magazine interview with UC Berkelely Haas Business School professor Philip Tetlock regarding “Why experts missed the crash.”
Having spent his career studying the ability of experts in variety of fields to make accurate predictions, Professor Haas was not surprised that most financial experts failed to call the Crash of 2008. His research shows that expert predictions:
- Often barely beat random guesses;
- Are seldom better than “reasonably well-read non-experts”, and
- Are even less accurate for the more famous experts.
Experts have the same trouble as the rest of us in accounting for the randomness that affects most situations. Also, their models are too simplistic for the complex world, and not even stellar credentials can make up for these deficiencies. But Professor Tetlock has found one common trait among the better forecasters:
The most important factor was not how much education or experience the experts had but how they thought. You know the famous line that [philosopher] Isaiah Berlin borrowed from a Greek poet, “The fox knows many things, but the hedgehog knows one big thing”? The better forecasters were like Berlin’s foxes: self-critical, eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability. The less successful forecasters were like hedgehogs: They tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything. The media often love hedgehogs.
Further, to tell whether an expert is a fox or a hedgehog:
Count how often they press the brakes on trains of thought. Foxes often qualify their arguments with “however” and “perhaps,” while hedgehogs build up momentum with “moreover” and “all the more so.” Foxes are not as entertaining as hedgehogs. But enduring a little tedium is worth it if you want realistic odds on possible futures.
Confidence certainly has its place — on the athletic field, on the dance floor, or when making a sale. But when peering into the future and trying to manage probabilities and even tilt them slightly in your favor, confidence alone just gets in the way. Better to emphasize curiosity and a sense of modesty when it comes to attempting a differential diagnosis of the unknown.
{ 0 comments… add one now }