Photograph by Kenishirotie / Shutterstock

The conventional wisdom of how most of us should invest our money is clear—avoid paying high fees to money managers for their supposed stock-picking expertise. In fact, steer clear of single stocks altogether, and simply buy “the market,” meaning an exchange-traded or mutual fund that passively tracks the performance of the entire stock market. And, maybe most important, focus on the long run, by holding investments through their ups and downs rather than trying to time the market by buying low and selling high—too tricky to do, say the experts.

This is good advice, as far as it goes. (And, for what it’s worth, I follow it myself, mostly.) On the other hand, a big pillar supporting it is the “efficient markets hypothesis,” economist-speak for the assumption that the prices of tradable assets like stocks, bonds, and commodities respond immediately and appropriately to new information, an assumption that depends on market participants, in other words people, acting rationally. Here, “acting rationally” means maximizing one’s return for a given level of risk, something economists call “mean-variance optimization.”

This hypothesis, if true, makes stock picking, timing highs and lows, or any other technique, powerless to beat the recommended strategy of simply buying and holding “the market.”

On the other hand, we know that some small percentage of people (think Warren Buffett, and a handful of other superstars) do regularly beat the market. And that the efficient market hypothesis’ picture of people as perfectly rational robots is, at best, an approximation that conflicts with the reality of irrationally exuberant rallies, market crashes in the absence of bad news, or any of Keynes’ animal spirits that all too often drive human behavior and with it that of the markets.

“Put less money into equities when markets are freaking out and leave money in equities when markets are more normal.”

In his new book, Adaptive Markets: Financial Evolution at the Speed of Thought, M.I.T. finance professor Andrew Lo attempts to account for the messier, more feeling realities of human behavior. A key premise is that markets evolve, like species, but much faster: “evolution at the speed of thought.” And that this evolution happens in fits and starts, in response to changes in the environment—hence, what he calls the “adaptive” markets hypothesis. It’s during these times of change that human emotions play their biggest role. Lo believes we are in one of those times now and, in his book, he applies biology, psychology, neuroscience, and history toward the goal of improving on the efficient markets hypothesis—which, Lo says, is not only flawed but is becoming increasingly so as the financial environment continues to change.



I spoke to Professor Lo on the phone one recent evening and he fervently held forth on these ideas. He thinks they support a new conventional wisdom around investing that is more in line with the realities of human behavior.





Why is the efficient markets hypothesis becoming more flawed over time?

Over the last few decades, the environment in which financial markets operate has been changing more rapidly than before. We have a larger population of financial market participants. By larger population, I mean two things. One, the world population has grown, and secondly globalization has allowed capital flows to occur more seamlessly between countries. We have French investors investing in Europe’s residential real estate during the financial crisis because the mutual funds and international holdings and capital flow restrictions are being eased; you’ve got a changing set of species that are participating in financial markets. You also have improvements in technology, so that the world is now much more globally connected than it was before thanks to social media, telecommunications, and so on.

That’s one of the reasons the approximation errors of the efficient markets hypothesis are growing. The hypothesis isn’t wrong, it’s just incomplete. It’s not the complete picture of how humans behave and how they interact with each other in financial markets. My assumption had been that maybe markets are not that efficient at some points, but things should be getting more and more efficient over time. Right? That’s what we learn about how competition works. It wasn’t until I started looking at the data over long expanses of history that it became clear to me that, very much like evolution, it’s not necessarily the case that species get more and more adapted to a given environment. That’s only true if the environment doesn’t change.

But it does change. The exceptions to the efficient markets hypothesis really come about when its standard assumptions are violated. The assumptions include things like stationary business environments, where the risks are relatively well known and humans act relatively rationally, at least from a mean-variance optimization perspective.

How does the adaptive markets hypothesis address the shortcomings of the efficient markets hypothesis?

It’s actually pretty straightforward. What I lay out [in the book] is basically what you would guess if you took the basics of evolutionary theory and extended them to the very special circumstances of financial markets. The efficient markets hypothesis is a special case of adaptive markets. Markets are efficient if the environment is stable and investors interact with each other and natural selection operates over a long period of time.

For example, the great white shark is pretty much in the same form today as it was in the very early days of the Pleistocene era. Part of the reason is that, in its environment, there’s no need for changes to its basic structure, given how well it has adapted. Now, imagine if you take that great white shark and you change the environment in substantial ways, either by changing the temperature of the water or changing the background colors so that the shark is much more visible than it might otherwise be. You’re going to change the ability of that shark to succeed. The implication is that an efficient market, [like the great white], is really an outcome of a very special set of circumstances.

How does irrational behavior enter financial markets?

A good example is the fight-or-flight response. Our evolutionary process hasn’t been able to catch up to the current threats we face, so it’s not surprising that the fight-or-flight response is not going to be the most helpful way to deal with a financial crash in the same way that it’s going to be helpful when you’re being attacked in a back alley. It’s because financial markets and financial threats are a relatively new phenomenon. The fact that when we are stressed financially, we end up exhibiting the same features as if we were threatened physically—that’s an example where human evolution has failed us, but not because there’s anything wrong with that evolutionary pathway.

How is this relevant for the individual investor?

One of the implications is that the risk-reward trade-off that we constantly bombard investors with, and constantly castigate them for ignoring when markets drop, has to be tempered by the acknowledgement that, sometimes, people will react emotionally to large-scale risk exposures. In technical terms, they freak out. When investors freak out, it means that they’re going to be pulling money out of the risky assets [like stocks] and investing them in much safer assets [like savings accounts and CDs].

If I’m right that investors do freak out from time to time, then the question that you might want to ask, during periods where investors are freaking out, is: Is it a good idea to hold stocks, or to put your money in cash during those periods, and then wait until the freak-out factor subsides?

What’s the better strategy?

From the behavioral finance perspective, and certainly from the data perspective, we find it’s the latter. When investors are freaking out, those are the periods where the equity premiums [meaning the extra return stocks should, by the efficient markets hypothesis, give investors as compensation for the extra risk of holding them in lieu of safer assets] are lower than average. In fact, in some cases it’s negative, you get punished for taking risks. The idea behind “volatility cruise control” is to maintain a level of volatility that is comfortable for an investor. Let’s say 16 percent volatility. That’s what the S&P is on average over a long history. If that’s the volatility that you’re comfortable with, then by using modern trading tools, we can get you that volatility actively.

In other words, if the stock market begins to increase in volatility—the VIX, [an index published by the Chicago Board Options Exchange that indicates how volatile traders expect the stock market to be over the next 30 days], goes up to 25 because there’s some kind of a rumor that the tax proposal that the Republicans are putting together isn’t going to work—the volatility cruise control strategy will reduce your level of equity exposure and put more money in cash. Similarly, when the volatility goes back to normal, you will then go back to putting most of your money into equities. That very simple rule of thumb basically has you putting less money into equities when markets are freaking out and leaving money in equities when markets are more normal.

That flies in the face of the conventional, buy-and-hold, wisdom—to sit tight and not freak out during market corrections, to ride them out.

That’s like telling a teenager, “It’s really a good idea for you to abstain from sex because a teenage pregnancy is just not going to be good for you.” It’s good advice, but I can tell you right now it’s not practical. Telling investors, “You know what, you should ignore these short-run dips and focus on the long run,” is good advice, but it ignores human behavior. From October 2008 to January 2009, if you had left your money in the S&P you would have lost 50.9 percent of your wealth. Slightly over half of your wealth would have evaporated in a manner of four months.

You tell me: How many investors do you know who would be perfectly happy and calm about watching half of their investment evaporate before their very eyes, while at the same time listening to news reports about Lehman going under, about Bernie Madoff in December of 2008, about financial markets coming to a halt, about Hank Paulson showing up on TV with a frightened face? If you think about how humans react, the advice that we give them—while it may be good advice if we really, truly stuck to it—it’s not realistic to expect humans to act in that way. That’s really the failing of the standard approach to passive investing. It’s advice that we know for a fact people are not going to take.

Financial innovations such as derivatives and securitization have been widely maligned for playing a role in the last crisis. But you argue that they could be the key to solving some of society’s biggest challenges, including poverty, disease, and climate change. Why?

People respond to incentives, and so if we want to take on much bigger challenges, we need to collaborate across thousands and in some cases hundreds of thousands of people. How do you get 100,000 people to work together? It’s not that easy. In the old days, it was religion and before that it was simple fiat rules, tyranny. The Egyptians built some beautiful pyramids, but they did that with hundreds of thousands of slaves over decades. If we rule out slavery as a possible means of societal advances, there really isn’t any other choice. If we need 100,000 people to cure cancer, to deal with Alzheimer’s, to figure out fusion energy and climate change…I don’t know of any other way to do that other than financial markets: equity, debt, proper financing and proper payout of returns. I think that in many cases [finance] probably is the gating factor. That, to me, is the short answer to the question about why finance is so important.



Bob Henderson studied physics, worked on Wall Street, and is now an independent writer focused on science and finance.



Get the Nautilus newsletter The newest and most popular articles delivered right to your inbox!

WATCH: The economist Robert Frank on why his colleagues don’t acknowledge how the environment influences us.