Not long after Alan Greenspan stepped down as Federal Reserve chairman in 2006, global financial markets began to unravel. The collapse of a few financial institutions, the near-collapse of many others, a massive bailout by multiple governments, and the worst economic downturn in three-quarters of a century ensued. Lots of people blamed Greenspan for some or all of this, and the man himself famously allowed, in a Congressional hearing in October 2008, that he had “found a flaw” in his model of how the world works.

Then, somewhat remarkably for a guy who had recently turned 80, he actually set to work trying to figure out what he’d gotten wrong. “It seems sort of silly,” he says now, “but I’ve learned more in the last three or four years than the previous ten.” In the fall he came out with a book setting out his new and improved worldview, The Map and the Territory: Risk, Human Nature, and the Future of Forecasting. But even that’s not the end of it. “I guess they shut it down in June,” he says, estimating when the book went to the printer. “I’ve come a long way since then.”

The book is not the world’s most coherent. It combines deep thoughts about the nature of economic forecasting with some pretty conventional ones about the dangers of entitlement spending without a clear framework or narrative to link them all together. The same goes for the new ideas Greenspan has been coming up with since June. But he nonetheless has some interesting things to say, most of them revolving around the idea that financial-market behavior isn’t fully rational, which makes the job of a central banker more complicated than envisioned back in the glorious 1990s. The boom-bust tendencies of Wall Street mean we need tougher capital requirements for banks, Greenspan now says, and maybe even a forced return to the partnerships that once dominated investment banking.

I tried to get Greenspan to talk me for my November HBR article on economics and finance since the crisis, but he said he’d promised his publisher to keep mum until the book was out, which was too late for my purposes. I finally did pay a visit to his office in D.C. late last year; what follows are heavily edited excerpts from our conversation. It would be inaccurate to call it an interview. I came armed with lots of questions, but ended up only getting to two or three. The man was in the mood for a monologue, which I have organized by topic.

Why Fed chairmen can’t do research

Greenspan had a long career as a private economic analyst and forecaster behind him when he was appointed Federal Reserve chairman in 1987. Delving into economic data is something he loved (and loves) to do, but at the Fed he says he didn’t get to do nearly as much of that as he would have liked.

It seems sort of silly, but I’ve learned more in the last three or four years than the previous ten. The trouble is if you’re at the Fed you can’t do a lot of research. There’s a funny anecdote I will tell you about quickly. I had an idea of constructing a certain statistical procedure. I called in a bunch of the senior analysts and I say this is what I have in mind. They all said, “Oh, it’s a terrific idea. Let’s do it.”

I said, “I’m going to do this one.” Silence. One guy says, “You’re CEO. You’re a chairman. You do CEO/chairman stuff. We do the research here.” I mean, I was really put down.

Fear beats greed

Greenspan was never a hardline believer in the rationality of financial markets. But his experiences and his studies since 2008 have convinced him that the behavioral component of market behavior is stronger and more predictable than he previously thought. For a guy frequently associated intellectually with his late friend and mentor Ayn Rand, he sure quotes John Maynard Keynes (mainly the famous phrase “animal spirits”) a lot.

The data show unequivocally that fear is a far more dominant force than euphoria. And it shows up everywhere. It shows in the business cycle — when you go into a crash it happens very quickly, when you recover it goes up very slowly. That is true of the unemployment rate. It’s true of GDP. It’s true of all commodity markets.

If you translate it into a probability distribution, it’s a more squatted one with significant bulge in the tails, but a far larger negative tail. Where all of us who set up models ran into a big mistake is we assumed that that the animal spirits essentially for analytical purposes were random. They are not. And one of the focuses of this book is to demonstrate there is a very considerable amount of systematic, asymmetrical biased behavior.

Bubbles and forecasting

The technology-stock bubble of the late 1990s and its subsequent deflation were among the defining events of Greenspan’s tenure. Greenspan never denied that financial bubbles exist, as witnessed by his famous “Irrational Exuberance” speech in December 1996. But he remains unconvinced that anyone can reliably predict when they will pop.

You can spot a bubble. They’re obvious in every respect. But it is impossible for the majority of participants in the market to call the date when it blows. Every bubble by definition deflates. But when that deflation occurs, it requires a point at which the vast majority of market participants do not expect it to happen. Almost everybody is bullish, expects the market to go up, and is fully committed. At that point if you took a survey of what the outlook was, you’d get an overwhelming positive response the day before it falls on its face.

That tells you a great deal about forecasting. This is the reason why everybody missed September the 15th, 2008.

One of the things that people don’t recognize is I made a speech in 2005, my last speech at the Jackson Hole conference, in which I was afraid to put a line in, so I put it in somewhat Fed-speak. I said, “Let’s remember that history has not dealt kindly with very prolonged low [risk] spreads.” And I said, “Do I dare dare say that?” And I said, “Yeah, it’s balanced enough. ”

I can recall a lot of people at the Federal Reserve raising all of these issues. I was sitting there 18-and-a-half years, getting an extraordinary amount of advice from everybody under the sun. Every day for most of the period from let’s say late 1980s basically through the end of my term, I would get almost every week people predicting that the world was coming to an end. That the economy was going to crash. “There are imbalances. There’s too much debt. There’s too much speculation.”

After a while you begin to say that this stuff is random, because you have this same thing going on on the other side. But in retrospect, what everybody reads is five or six guys who did get it right.

You know, when [Eugene] Fama and [Robert] Shiller got the Nobel, in the New York Times, they have Fama saying about Shiller’s forecast of a decline in housing prices, “Aw yeah, he’s been saying that for years.” [Greenspan asked me to check that, and the actual line from the article was pretty close: ”Asked in 2010 about those who warned that housing prices would crash, he responded, ‘Right. For example, Shiller was saying that since 1996.'”]

Even if you believe it, you shouldn’t say that. I mean it’s un-nice in the extreme. But the problem is it’s true. I had at my call 250 first-rate PhDs in economics. I mean good ones. They’re building this thing we called FRB/US, this extraordinarily clever set of the most advanced econometrics that you can imagine. With every bell and whistle that anybody suggested, it was in there. It missed the crisis.

The IMF missed the crisis. My friends at J.P. Morgan put out a weekly forecast. They’ve been doing it for years. And three days before September the 15th, they had the economy going straight up.

What I’m trying to get at here is that implied in a lot of the stuff that’s being written now is that there were people who actually really got it right. And the answer is yes, that is factually accurate. But you would not have made money off their predictions in the past. I know these people.

Why it’s hard to pop bubbles, part 1

Over the space of a few years, Greenspan went from being hailed as the near-infallible Maestro to being blamed for pretty much everything bad that ever happened to the economy. It’s clear that he thinks he’s gotten both too much credit and too much blame, but he has also developed an interesting theory – that good central bank performance actually breeds bubbles and crashes.

I used to discuss the issue at the Fed, “What do we get by being very successful in forecasting?” And I say a bubble. In fact I think the evidence probably is conclusive that a necessary and sufficient condition for a bubble is a prolonged period of economic stability, stable prices, and therefore low risk spreads, credit-risk spreads.

Well the question is, do you quash the bubbles? The answer’s yes, you can quash them, but not in a democratic society. I mean, I hesitate to think what would have happened if the Federal Reserve had tried to quash the dot-com boom — assuming we knew exactly what it was all about, which we didn’t, until later. All hell would have broken loose.

The Federal Reserve’s independence would have been severely constrained, believe me. We were independent in a sense that no other agency of government can countermand the actions of the Federal Open Market Committee. But they can change the law.

All sorts of threats were always there. One, for example, is to take the voting power away from the [Federal Reserve Bank] presidents, who statistically are demonstrably more hawkish than the politically appointed governors. All sorts of things like that could occur.

I’m surprised the Fed’s independence wasn’t more significantly curtailed in this post-crisis era. It was curtailed slightly. The nature of the boards of the individual 12 Reserve Banks was altered, but not enough to fundamentally affect policy as far I’m concerned.

If the Federal Reserve unanimously concluded that there was a major bubble underfoot and if we don’t stop it now within five years it would do in the country, the Federal Reserve could not move. It would be moving in a way which would create a countervailing political force. And we were aware of it. We never assumed that we were politically independent in the true sense of the word.

In the 18-and-a-half years that I was there, I got a huge number of letters, or notes or whatever, [urging] us to lower interest rates. On the side of getting letters which say you’ve got to tighten [raise rates], it was a zero. So there’s a huge political, regulatory asymmetry.

This is the political bias that stems from the asymmetry of human nature. If everyone were wholly rational in their long-term self-interest, there would be no political biases. We’d all be perfect people, including regulators.

Why it’s hard to pop bubbles, part 2

William McChesney Martin, the only Fed chairman to serve longer than Greenspan, famously said that a central banker’s job is to take the punch bowl away just as the party starts warming up. Greenspan, however, now wonders whether calming down a financial-market party might actually be impossible. In this view, the “soft landing” engineered by the Fed in 1994-1995, hailed at the time as a masterpiece of monetary policy-making, may have actually set the stage for the wild times that ensued.

We at the Fed kept being concerned about bubbles. We didn’t call them bubbles at the time. We called them booms.

Is there anything you can do to stop a boom? We actually tried to do that [before] the dot com boom. For the first time we got a so-called safe landing. As soon as we let up, off it went, which suggests to me that what happened was that because our actions failed to knock the economy down despite a three-percentage-point increase in the federal funds rate, it elevated the long-term asset value expectation.

If that is true, and I raise it as a hypothesis because I don’t know how you could prove it one way or the other, it is saying that our very efforts made the situation worse. And the notion that you could calibrate monetary policy to suppress a boom against the human nature bubbling up has no factual basis. The only place it exists is in the econometric models where the equations are so constructed that if you tighten monetary policy you smooth out the boom.

That is not the way that the markets essentially work. So, the question is if you can’t stop a bubble, what do you do? Fortunately most bubbles are not toxic. The dot-com boom when it collapsed, you can’t find it in the GDP figures in 2001, 2002. It didn’t happen. The 1987 crash, which was really the most horrendous thing, I defy you to find that in the GDP numbers. It’s not there. Yes, there were huge capital losses, but to the people who made the capital gains, essentially.

[I scratched my head a little at that statement about 2001-2002, since there was a recession in those years. When I followed up with Greenspan, his response was: “Look at the data. It’s remarkable how small it was, and with revisions it may eventually disappear.”]

The same is true of a lot of bubbles. It was not true of the housing bubble because of debt. It wasn’t the nature of the asset, namely subprime mortgages, which was critical. It was the form in which the asset was financed. If the people who owned mortgages in one form or other were wholly unleveraged, there would be a lot of losses but there would not have been the contagion.

In other words, a necessary condition for a bubble to be toxic is debt or leverage. That then leads to the fascinating question, “Well if you can’t stop a bubble, what can you do to arrange that it’s benign?”

We’re starting to do things. I mean for example we’re doing higher capital [requirements], but we’re also delving into CoCo* bonds and in principle they can fully defuse anything. Obviously they’re more costly to issue. But that’s the true cost. People say, “Well, it’s more expensive.” It should be more expensive because there is a free ride that’s been going on.

* “CoCo” is short for contingent-convertible — a bond that automatically converts into stock if some pre-set threshold (a bank’s capital ratio drops below a certain level, for example) is breached.

Cleaning up after a bubble bursts

The Fed’s decision to keep short-term interest rates below 2% for three years after the dot-com bust has been held up again and again as a possible cause of the subsequent real estate bubble. Greenspan doesn’t buy that, as he explains below, but perhaps more interestingly he also says that keeping rates low would have been the right thing to do even if it had led to real estate excesses.

Look, we were and we still are pilloried for lowering the federal funds rate in 2003. What I was doing was saying, “What is the probability of deflation?” I would say that the probability was probably 20%. But what is the consequence should it happen? And I’d say horrendous.

So, in that context you come up with the conclusion yes, it’s an improbable event, but it still requires insurance. In the same sense that fire insurance is taken out. We were acutely aware that the decline in the federal funds rate was more than likely to be unnecessary. But we took out the insurance nonetheless because of the asymmetry of the potential outcomes. Now, that’s the way I think.

I’ve said given the same circumstances we would have done it and should have done it again. Now, in fact it turned out that there is no evidence that that affected the money supply, the five-year Treasury note yield, or any of the other elements which should be characteristic of engendering a housing boom. As a consequence it could have but it didn’t have a negative effect.

Why financial firms shouldn’t be corporations

For his book, Greenspan dug up an 1863 quote from the first Comptroller of the Currency (the regulator of national banks in the U.S.), Hugh McCulloch, who said the officers and directors of a bank should “be made personally liable for the debts of the bank” if it failed. That never happened, but the partnerships that came to dominate investment banking did stick partners with personal responsibility for the partnership’s debts. It was only in the 1970s that the Wall Street partnerships began converting to limited-liability, publicly traded corporations, and Greenspan now thinks that may have been a big mistake.

The thing I’ve been thinking about most recently is, “Is the corporation the ideal vehicle for finance?” And the answer is no.

I make a point in the book of the 1970 change in New York Stock Exchange regulations with respect to the ability of broker/dealers to be able to incorporate. One of my clients was Donaldson, Lufkin & Jenrette. They were the first to incorporate. I lived through it at the time; I didn’t have a clue what I was observing. The long-term implications of that I certainly didn’t perceive, but in retrospect it’s sort of fascinating. My very first client when I was a sophomore in college was Brown Brothers Harriman, which was a big deal at that time. I remember I went down to their offices for the first time and I when I got inside the door I sunk in six feet of carpet.

They’re still around. I recently went back and I looked. I knew that they remained a private bank. They obviously missed the boom. They missed a lot in the beginning. From being a relatively very important player in Wall Street for a long time, they faded as they did not participate in the shenanigans.

The reason essentially is the partnership. I remember, not Brown Brothers as I wasn’t that close to them, but I remember companies whom I did get very close to as clients: Bear Stearns, Lehman. They wouldn’t lend a dime to you overnight when they were partnerships. The partners themselves were very chary of their own equity.

Everyone is looking at the mistakes that the people at Bear and Lehman made. None of them went on food stamps or anything like that. There’s something wrong in financial intermediaries which we’re not really confronting, which is one of the issues I’m getting at.

There are two fundamentally different types of organizations in a market economy. One is finance and the other is non-finance. The non-financial sector in the economy leading up to the 2008 crisis was in as good a shape as one could imagine. They basically had long-term debt and equity easily matching their long-term illiquid assets. Their amount of short-term debt to long-term debt was very low. Most of the capital investment decisions were based on rational data.

What I’m trying to get at is that the vast proportion of decisions that are being made by non-financial institutions are based upon objective reality and its probability structure. Finance, however, is almost wholly behavioral animal spirits. It’s a different type of paradigm.

How to beat the market

Greenspan may not think central banks should try to outsmart financial markets, but he does think investors can profit from the tendency toward bubbles and, in particular, crashes.

One thing about markets is that they may not be Fama’s view of the way of the world, but it’s not easy to make money. The only stock market procedure that I’ve found makes money rests basically on the the asymmetrical nature of probability distributions, the fear-euphoria issue.

That tells you why somebody like Warren Buffet does so well. I mentioned to Warren a while back, I said, “Warren, it strikes me that if you did nothing else you never sell.” That is, if you can grit your teeth through and just disregard short-term declines in the market or even long-term declines in the market, you will come out well.

I mean you just stick all your money in stocks and go home and don’t look at your portfolio you’ll do far better than if you try to trade it. The reason that’s the case is this asymmetry between fear and euphoria. The most successful stock market players, the best investors, are those who recognize that the asymmetric bias in fear vs. euphoria is a tradable concept and it can’t fail for precisely this reason.

So there are stabilities here which are very important, but along with them are more junk statistics, more junk analysis, and more stock market letters than should be allowed to be written. It’s just sort of ridiculous.