By now you will probably have read an awful lot about the financial crisis. Perhaps I've been reading all the wrong stuff, but until now I hadn't managed to find answers to the most puzzling questions. If the crash of 2008 was preceded by an era of unprecedented prosperity, how come most of the people I know weren't earning much?

Deregulation of financial services was supposed to have made us all better off, so why did most of us have to live off credit to keep up? Now that it has all gone wrong, and everyone agrees we're in the worst crisis since the Great Depression, why aren't we following the lessons we learned in the 1930s?

President Obama attempted one of the biggest Keynesian stimulus programmes. Why has it been only minimally effective? Why do most other western leaders still insist the only way out is to tighten our belts and pay off our debts, when that clearly isn't working either? And how come the bankers, credit agencies and bond traders are still treated with cowed reverence – don't frighten the markets! – when they got us into this mess?

These mysteries were beginning to make me feel as if I must be going mad – but since reading Paul Krugman's new book, I fear I'm in danger instead of becoming a bore. It's the sort of book you wish were compulsory reading, and want to quote to anyone who'll listen, because End This Depression Now! provides a comprehensive narrative of how we have ended up doing the opposite of what logic and history tell us we must do to get out of this crisis.

Its author is a Nobel prize-winning economist who writes a column in the New York Times and teaches economics at Princeton University. An authority on John Maynard Keynes, Krugman wrote a book in 1999 called The Return of Depression Economics, largely about the Japanese slump, which drew ominous parallels between Japan's economic strategy and the pre-New Deal policies of the early 30s that turned a recession into catastrophic depression. At the time, unsurprisingly, most western economists weren't bowled over; in thrall to the seemingly endless boom, the Great Depression looked to them to be more or less irrelevant. Krugman's latest book will be much harder to ignore.

He doesn't expect it will be an easy message to sell, though. "As far as I can make out, the serious opposition to the coalition's policy is basically a half-dozen economists, and it looks as if I'm one of them – which is really weird," he laughs, "since I'm not even here." Visiting London last week, he met lots of what he calls Very Serious People: "And there are lots of things these people say that sound very wise and sensible. But it's all upside-down; it's all wrong. Yet the power of their orthodoxy – even when it's failing – is quite awesome."

These Very Serious People present economics as a morality play, in which debt is a sin, and we have all sinned, so now we must all pay the price by tightening our belts together. They tell us the crisis will take a long time to resolve, and must inevitably be painful. All of this, according to Krugman, is the opposite of the truth. Austerity is a self-imposed collective punishment that is not just unnecessary, but won't work. We know what would work – but for complex political and historical reasons that his book explores, we have chosen to forget. "Ending this depression," he writes, "should be, could be, almost incredibly easy. So why aren't we doing it?"

Krugman offers the example of a babysitting co-op, or circle, in which parents are issued with vouchers they can exchange for babysitting hours. If all of the parents simultaneously decide to save their vouchers, the system will grind to a halt. "My spending is your income, and your spending is my income. If both of us try to slash our spending at the same time, then we are also slashing our incomes, so we don't actually end up saving more." We could issue more vouchers to everyone, to make them feel "richer" and encourage them to spend – which would be the circle's equivalent of quantitative easing. But if everyone is determined to save, the parents will hold on to the extra vouchers, and the circle still won't work. This is what's called a liquidity trap, "and it's essentially where we are now".

The same principles apply to the "paradox of deleverage". Debt in itself is not a terrible thing, he says. "Debt is one person's liability, but another person's asset. So it doesn't impoverish us necessarily. The real danger with debt is what happens if lots of people decide, or are forced, to pay it off at the same time. High debt levels make us vulnerable to a crisis – and this is when you get the self-destructive spiral of debt deflation. If both of us are trying to pay down our debt at the same time, we end up with lower incomes, so the ratio of our debt to our income goes up."

Crucially, Krugman continues, "what's true for an individual is not true for society as a whole". The analogy between a household budget and a national economy is "seductive, because it's very easy for people to relate to", and it makes some sense when we're not in the grip of a macro-economic crisis. "But when we are, then individually rational behaviour adds up to a collectively disastrous result. It ends up that each individual trying to improve his or her position has the collective effect of making everybody worse off. And that's the story of our times."

At these moments someone has to start spending – and, Krugman argues, it is the government. But we're endlessly being told by the coalition that it has to pay off its debts because servicing the interest is ruinous, and the bond markets will destroy us unless we're seen to be tackling the deficit.

"Well, now. We know that advanced economies with stable governments that borrow in their own currency are capable of running up very high levels of debt without crisis. And we know it, actually, best of all from the history of the UK – which spent much of the 20th century, including the 30s, with debt levels much higher than it has now."

But what about bond markets? Invoked as global bogeymen, we're warned that they punish governments who fail to cut spending – even if cuts don't reduce the deficit. I've never understood why the markets should care how and when we reduce the deficit, as long as we can pay our way. According to Krugman, they don't.

"That's the interesting thing. The actual verdict of the markets, for countries that have their own currencies, has been that they don't really care at all in terms of what you're doing in short-run policy." Likewise, the danger of being downgraded by a credit rating agency has been wildly overstated. "We saw it in Japan in 2002; they had the downgrade, and nothing happened. Which led us to predict that would happen for the US," whose credit rating was downgraded by one agency last year. "And it was exactly right. Nothing happened."

A breadline in the US in 1930. According to Krugman, our governments have failed to learn the lessons of the Great Depression. Photograph: American Stock Archive/Getty Images

Thus far, Krugman has essentially restated the case for Keynesianism. "And these are not hard concepts, actually. It's not hard to get it across to an audience. But it doesn't seem to play in the political sphere." What's fascinating is his historical analysis of why policy-makers, who once understood these principles, collectively decided to forget them.

In the years following the Great Depression, governments imposed regulatory rules upon the banking system to ensure that we could never again become indebted enough to make us vulnerable to a crisis. "But if it's been a long time since the last major economic crisis, people get careless about debt; they forget the risks. Bankers go to politicians and say: 'We don't need these pesky regulations,' and the politicians say: 'You're right – nothing bad has happened for a while.'"

That process began in earnest in 1980, under President Reagan. One by one the regulations on banking were lifted, until "we lost the safeguards, and it meant there was an increasingly wild and woolly financial system willing to lend lots of money". Politicians were in part persuaded to deregulate by the argument that it would make us all richer. And to this day, "there's this very widespread belief that there was, in fact, a great acceleration in growth. But this really isn't hard. You sit down for a minute with the national account statistics, and you see it ain't so."

If we divide the period between the second world war and 2008 into two halves, "the first half is a really dramatic improvement to living standards, and the second half is not." It was certainly dramatic for the top 0.01%, who saw a seven-fold increase in income; in 2006, for example, the 25 highest-paid hedge fund managers in America earned $14bn, three times the combined salaries of New York City's 80,000 school teachers. But between 1980 and the crash, the median US household income went up by only roughly 20%. "So it's a total disconnect."

Why would economists claim ordinary people were getting much richer if they weren't? "The answer, I think, has to be that you need to ask: 'Well who are the people who say these things hanging out with? What is their social circle?' And if you're a finance professor at the University of Chicago, the people that you're likely to meet from the alleged real world are going to be people from Wall Street – for whom the past 30 years have, in fact, been wonderful. If you're a mover and shaker in the UK, you're probably hanging out with people from the City. I think that is the story of the disconnect."

But the influence of the top 0.01%'s mindboggling wealth didn't stop at finance professors. Their mansions and yachts and luxury lifestyles created "expenditure cascades", whereby, "if you're a little bit down the income distribution from there, you're going to feel some compulsion to match some of that too. And then, in turn, the people below you can feel some compulsion too."

There were early warning signs, such as the savings and loans crisis of the late 80s, that should have alerted politicians to the dangers of financial deregulation, moral hazard and subsequent spiralling debt. But by then Wall Street's influence over policy-makers had rendered them deaf to alarm bells – in part because bankers were financing so many politicians' campaigns. Krugman quotes Upton Sinclair's famous observation: "It's difficult to get a man to understand something, when his salary depends on his not understanding it" – but more than that, he suspects the sheer glamour of wealthy bankers had a powerful influence over politicians.

"My impression is that old style captains of industry can be rather boring. I'm not sure how much thrill there is in hanging out with someone like that. But Wall Street people are in fact very smart; they're funny, they're not company men who work their way up the chain. They're impressive."

Even Obama is not immune to their charms, says Krugman. Early into the administration he met the president and his economics team, "and it was just clear that rumpled professors with beards just didn't come across as being so impressive. Yeah," he chuckles. "I had that definite sense." But even many of the rumpled professors had been seduced by the promise of a new world economic order, in which Keynesianism was not just redundant but faintly ridiculous.

By 1970, Krugman writes, "discussion of investor irrationality, of bubbles, of destructive speculation had virtually disappeared from academic discourse. The field was dominated by the 'efficient-markets hypothesis'," which persuaded economists that: "We should put the capital development of the nation in the hands of what Keynes called a 'casino'." The death of Keynesianism was "triumphantly" announced, largely by Republican economists whose work had become "infected by partisanship and political orientation". Now, as they are faced with the catastrophic collapse of their theories, Krugman thinks political bias and professional pride are what's stopping them admitting they were wrong.

Those economists cite the woefully limited impact of Obama's almost $800bn stimulus package as proof that they are still right. According to Krugman, the only thing wrong was it wasn't enough. Almost half went on tax cuts, and most of the remaining $500bn went on unemployment benefits, food stamps and so on. "Actual infrastructure spending – that's more like just $100bn. So if your image of the stimulus programme is: 'We're going out there and building lots of bridges' – that never happened."

In an economy that produces $15tn worth of goods and services each year, $500m "is just not a big number". Back in 2009, Krugman had warned: "By going with a half-baked stimulus, you're going to discredit the idea of stimulus without saving the economy." And that, he sighs, "is exactly what happened. Unfortunately it was one of those predictions that I wish I'd been wrong about. But it was dead on."

Since the crash Krugman has become the undisputed Cassandra of academia, but he jokes: "I'm kind of sick of being Cassandra. I'd like to actually win for once, instead of being vindicated by the disaster coming – as predicted. I'd like to see my arguments about preventing the disaster taken into account instead."

The likelihood of that is a fascinating question. Krugman is not the most clubbable of fellows. In person he's quite offhand, an odd mixture of shy and intensely self-assured, and with his stocky build and salt-and-pepper beard he conveys the impression of a very clever badger, burrowing away in the undergrowth of economic detail, ready to give quite a sharp bite if you get in his way. His public criticisms of the Obama administration have upset many Democrats in the US, while his more vociferous criticisms of George Bush used to earn him death threats from angry rightwingers.

I hope none of that gets in the way of his argument. What we need to do, Krugman says, is simple: ditch austerity, kickstart the economy with ambitious government spending, and bring down the deficit when we're back above water again. Most importantly of all, we need to do it now.

"Five years of very high unemployment do vastly more than five times as much damage as one year of high unemployment. To say: 'Yes, it's painful, but time does heal these things … " He breaks off and sighs in despair. "Well, no. Time may not heal it."

• This article was amended on 13 June 2012. The original said President Obama was the only world leader who had attempted a Keynesian stimulus programme in response to the 2008 financial crash. This has been corrected.