Illustration by Kevin Kallaugher

“DUMB and Dumber”, one of the modern classics of American comedy, tells the story of an affable idiot, Lloyd Christmas, who falls in love with a classy beauty, Mary Swanson. In one scene he asks her the chances of “a guy like you and a girl like me” ending up together. The answer is “Not good”. “Not good like one out of a hundred?” asks Lloyd. “More like one out of a million,” Mary replies. Lloyd pauses for a moment, then shoots back, “So you're telling me there's a chance?”

That is the American spirit. Americans have traditionally been much more optimistic than Europeans, and happier too. They believe that people determine their own destinies rather than being the mere playthings of fate. They also believe that their children will have a better life than they do.

This helps to explain the dynamism of the American economy. Americans regard failure—even bankruptcy—as a stone in the road rather than a damning verdict. It also helps to explain the appeal of the American experiment. “The American, by nature, is optimistic,” JFK once said. “He is experimental, an inventor and a builder who builds best when called upon to build greatly.” Europeans ask, “Why?” Americans ask, “Why not?”

But the past five years have produced a dramatic souring in the country's mood. Three-quarters of Americans now think that the country is “on the wrong track”. The number of people who are hostile to trade and immigration has risen sharply, as has the number who think that America should engage less in world affairs. Trust in government is half what it was in 2001. Both the president and Congress have bargain-basement approval ratings. And people are increasingly pessimistic about the future. Barely a third thinks that their children will be better off than they themselves are. The reason why the man with the message of hope, Barack Obama, and the man who hails from Hope, Mike Huckabee, are doing so well is that people think hope is in short supply.

There are good reasons for this pessimism. People are tired of the Bush administration, with its partisanship and incompetence. The Iraq war has shattered America's image of itself as a mighty but good-hearted giant. Iraq means chaos and Abu Ghraib rather than the triumphant export of democracy. The war may be going better of late, but the majority of Americans still think that it was not worth the fighting, and the bad news about the economy is burying the good news about the “surge”. Americans are accustomed to thinking of their houses as pots of gold (and pots of gold that can be remortgaged when you need a bit of extra cash). The gold is now turning into lead.

This is creating a bull market in pessimism. The likes of Bill O'Reilly and Lou Dobbs have transformed themselves into cable stars by ranting about cultural decay and “broken borders”. Patrick Buchanan's latest book is called “Day of Reckoning: How Hubris, Ideology and Greed are Tearing America Apart”. “We are on a path to national suicide,” he says. America is not just “coming apart”, but also “decomposing”.

There is certainly no shortage of bad news. But coming apart? Decomposing? The current issue of Commentary—a magazine hardly noted for its sunny disposition—contains an excellent article, “Crime, Drugs, Welfare and Other Good News” by Peter Wehner (a former senior White House aide) and Yuval Levin, which shows why Mr Buchanan is talking through his hat.

Both violent crime and property crime have declined dramatically since 1973. New York City will probably notch up less than 500 murders this year, the lowest since the early 1960s (the figure for 1990 was 2,262). Teenagers are cleaning up their act. Teenage drug use has fallen by 23% overall since the 1990s, and by 50% for LSD and ecstasy. Teens are drinking less, smoking less, having sex less and dropping out of school less. The birth rate for 15-19-year-olds has fallen by 35% since 1991. At 10%, the high-school drop-out rate is at a 30-year low.

Welfare reform is working. The welfare caseload has dropped by 60% since 1994. A series of social evils—overall poverty, child poverty, child hunger—have all decreased. Employment figures for single mothers have surged. The number of abortions fell from over 1.6m in 1990 to fewer than 1.3m this year. The divorce rate is at its lowest level since 1970. Education scores are up.

Things are not all hunky-dory. The violent crime rate has ticked upward in the past couple of years. Illegitimacy is at an all-time high. But this is a very different world from the 1970s, when America's leading cultural indicators all started to point towards Gomorrah. American civil society, aided by sensible social policies, is gradually repairing the damage of the past few decades.

Lighten up, Pat

What about Mr Buchanan's particular bugbear, immigration? For all the passion that this is igniting, America is blessedly free from the alienated immigrant populations that so trouble France and other European countries. America still has a genius for employing and assimilating newcomers. English classes are crammed and English-language tuition is a giant business. And America's ability to suck in and assimilate immigrants is part of a bigger and better story. European economies face profound long-term problems because of their citizens' reluctance to reproduce. Americans have the highest fertility rate of any rich country. That is a long-term vote of confidence in the country's future.

In fact, Americans have always had a vigorous tradition of pessimism, in counterpoint to the optimistic one. In 1819 John Adams worried about the duration of “our vast American Empire”. In 1948 Richard Hofstadter complained that “competition and opportunity” had gone into decline and that Americans were looking “wistfully back toward a golden age”. Much of today's pessimism may prove as unfounded. Americans should rediscover the spirit of Lloyd Christmas, idiot though he was.