Yale professor Robert Shiller, author of the new book Finance and the Good Society, says that historically home prices have not been a good financial investment -- they essentially tracked the rate of inflation from 1890 to 1990.

Yet during the housing boom of the 2000s, the mind-set of most Americans was that housing was a great capital-gains-generating investment. I recently asked Shiller, one of the leading thinkers on U.S. housing, how we collectively fell into that illusion. Hear his explanation in the video below, excerpted from my interview with him in front of a live audience at Motley Fool Headquarters. (Running time is 3:51; a transcript is provided below.)

Please enable JavaScript to view this video.

Robert Shiller: There's no guarantee that home prices are going to go up. I think we've gotten into an illusion about that. We got into an illusion and it created this spectacular bubble. We have to reflect now that we had a kind of crazy mind-set in the last couple of decades, and we have to get back to thinking like people used to think. Housing is a depreciating asset, goes out of style; it's going to end up in the wrong place. People will want to live somewhere else, so it's not any automatic capital gain.

Brian Richards: You've written extensively about investor behavioral psychology. How did we fall into that illusion and how do we work ourselves out of it?

Shiller: That's a very interesting question. How did we get this idea that home prices only go up? There are a number of elements of it. I don't know where to start. One of them is that we had a lot of inflation. I'm talking psychology now. You're asking how we got into a wrong view. In the 70s and 80s, we had a lot of inflation and then Paul Volcker came in and stopped it. So inflation has been declining now for 30 years, and we've lived our lives in that environment.

But we still encounter examples when someone says, "My grandmother just sold her house." Especially five years ago, say this happened five years ago. Grandma sold their house for $300,000, and do you know what she paid for it in 1952? It was only $30,000 or something like that. So it went up ten-fold. Now those stories are in all of our repertory, but when you really look at it, what was just consumer price inflation over that period? It was something like that. She really didn't make any money off of it. And she was putting money into it year after year and maintaining it. So we forget that. It's that kind of bias.

Also I think that we're influenced not by population growth so much, but by the sense of the growing wealth of the world and the finiteness of land, and we mistake land for…well that's another thing that happened. We started to think of urban real estate as land. And that's a change in our thinking.

If you go back hundreds of years, there was land speculation in this country, but there was no housing, not much urban housing speculation. So it was common sense. Talk to George Washington, if you could, all right? George Washington was a land speculator, and he owned Mount Vernon as among his speculations. But for George Washington, speculating in real estate meant buying thousands of acres for a shilling an acre or something like that. Not buying a house in the city, so we've changed. It's become much more proliferated as something that everyone does. You buy this house and it's going to make you a lot of money.

It's also just the bubble itself -- the Fed had very loose policy and that encouraged the bubble and prices were going up fast, so that proliferated stories about real estate as an investment. Anyway, that's a complicated analysis of our psychology. But it is a unique phenomenon, really, that it was so national. And it also reflects our better communications now. It wasn't as easily so national in the past.

For more insights from my talk with Robert Shiller, see: