Explaining why the market got so "crazy"

In the past week, two interesting articles about the financial crisis have pointed to the limits of mathematical modeling in finance and the role of group psychology in the market.

Yale economist Robert Shiller blames "groupthink" for why many economists were not outspoken about the housing bubble until it was too late. In explaining the phenomenon, Shiller says:

"The field of social psychology provides a possible answer. In his classic 1972 book, “Groupthink,” Irving L. Janis, the Yale psychologist, explained how panels of experts could make colossal mistakes. People on these panels, he said, are forever worrying about their personal relevance and effectiveness, and feel that if they deviate too far from the consensus, they will not be given a serious role. They self-censor personal doubts about the emerging group consensus if they cannot express these doubts in a formal way that conforms with apparent assumptions held by the group."

Shiller is a proponent of "Behavioral Economics," which looks at the underlying psychological factors behind our economic decisions. In this case, behavioral approaches can help explain the "irrational exuberence" people experienced in the housing market, which caused housing prices to jump far beyond what could be explained by economic fundamentals. Shiller also charges that behavioral economics is still considered a "fringe" field within the discipline and that groupthink makes it harder for economists to advocate behavioral approaches in public.

Steve Lohr also talks about another way that human error contributed to the crisis. Financial institutions were built on complex mathematical models of risk, which were supposed to prevent just these types of crises. Mathematical models are prevalent throughout economics and are truly invaluable tools in both theoretical and empirical work. But it's really important to recognize their limits:

"'The Wall Street models', said Paul S. Willen, an economist at the Federal Reserve in Boston, 'included a lot of wishful thinking about house prices'. 'But', he added, 'it is also true that asset price trends are difficult to predict. The price of an asset, like a house or a stock, reflects not only your beliefs about the future, but you’re also betting on other people’s beliefs,' he observed. 'It’s these hierarchies of beliefs — these behavioral factors — that are so hard to model.'"

Emanuel Derman, a physicist who developed a number of financial models, put it more simply:

“To confuse the model with the world is to embrace a future disaster driven by the belief that humans obey mathematical rules.”

The current crisis may cause more people to change their approach to modeling financial markets. I think it's likely that behavioral economics and social psychology will play a larger role in understanding why markets become so irrational sometimes. This is probably a good thing, which will add to our understanding of how financial markets work. But we shouldn't throw the baby out with the bath water. Emanuel Derman notes that these models are simply tools that can be used incorrectly or inappropriately. Complex financial models will retain their important place within the industry and in academia. But we need to do a better job understanding the assumptions behind these models and how they relate to real human behavior.