In a recent NY Times piece, noted imperial advisor Nicolas Kristof points out that the Emperor’s Advisers Have No Clothes:
The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.
The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.
I salute Kristof for having the courage to declare that his profession (along with pretty much everyone else) has no clothes (or at least far fewer clothes than we think—perhaps a German speedo!) I have sometimes disagreed vehemently with Kristof over various issues, but I have to admire anyone who has the moral courage to turn the harsh light on his own profession, and even himself:
The marketplace of ideas for now doesn’t clear out bad pundits and bad ideas partly because there’s no accountability. We trumpet our successes and ignore failures — or else attempt to explain that the failure doesn’t count because the situation changed or that we were basically right but the timing was off.
For example, I boast about having warned in 2002 and 2003 that Iraq would be a violent mess after we invaded. But I tend to make excuses for my own incorrect forecast in early 2007 that the troop “surge” would fail.
As Kristof notes, the greatest problem is with extremely self-confident experts who are morally certain that they’re right:
Mr. Tetlock called experts such as these the “hedgehogs,” after a famous distinction by the late Sir Isaiah Berlin (my favorite philosopher) between hedgehogs and foxes. Hedgehogs tend to have a focused worldview, an ideological leaning, strong convictions; foxes are more cautious, more centrist, more likely to adjust their views, more pragmatic, more prone to self-doubt, more inclined to see complexity and nuance. And it turns out that while foxes don’t give great sound-bites, they are far more likely to get things right.
Perhaps the problem is that expertise grants enormous ability to describe details about the present state of things in a field or area. There’s no absolute logical requirement for that to translate into better prediction in that area, but we’re basically hard-wired to think that there is. Combine that with our hard-wired response to confidence and we’ll follow a knowledgeable hedgehog anywhere, even to Hell itself.
Contrast that with the ability of markets,in general, to outperform experts. The best example is the stock market, whose simple average generally outperforms 2/3 of managed mutual funds each year (an example from last year: http://www.winninginvesting.com/index_vs_managed_funds.htm and another: http://www.fool.com/investing/mutual-funds/2009/02/25/index-funds-are-hard-to-beat.aspx). Over time, the results are even more staggering, as very few mutual funds repeat winning performance from year to year. Experts simply cannot complete with the “wisdom” of the market as a whole.
An important thing to keep in mind, especially today as we’re urged to put more and more confidence in expert management of the financial sector by government regulators, expert allocation of money by government bureaucrats, and the most massive transfer of money from the private market-driven economy to government experts in recent history. It’s perhaps no coincidence that the government is relying upon the least-regulated part of the financial sector (which has been least affected by the recent woes) to bail out the most-regulated part:
Democrats like Barack Obama and Barney Frank, at least on the campaign trail or in sound bites, have portrayed the financial crisis as the product of deregulation. The solution, they say, is more regulation. In that vein Frank, one of the brainiest members of Congress, is proposing that the Federal Reserve become a regulator of systemic risk, with the power to regulate firms that because of their size or strategic position are of systemic importance.
My American Enterprise colleague Peter Wallison has argued powerfully that this is a bad idea. Neither the Federal Reserve or other regulators identified the systemic risk which caused this crisis. Neither did most financial institutions or investors. Systemic risk is hard to identify for the very reason that it is systemic.
If experts are as unreliable as Kristof argues, can expert risk assessment (which is what regulation is) be expected to outperform market-based risk assessment (which is what an unregulated market does)? Certainly not all the time, which is precisely the lesson that we should learn from the recent crisis originating in the heavily-regulated mortgage industry…and precisely the lesson our leaders seem determined not to learn.
(Hat tip to Angry New Mexican for the Kristof link and article!)