A few weeks ago, in a large business hotel in Washington DC, a crowd of around 80 people had congregated outside one of the ballrooms when the double doors opened and a bald man in jeans and a lumberjack shirt backed out into the hallway. He was bent almost in half, wedging the doors open. "We're almost ready to begin!" he called out. The crowd, who were waiting for a panel discussion on the future of the American right, entitled Conservatism 2.0, paid only partial attention: the organisers had just started distributing sandwiches and cans of Coke, which interested them more. Then the man stood up, and brushed off his hands on his jeans. He smelled strongly of aftershave.

"Oh. My. God," someone said quietly.

Slowly at first, then faster, the realisation began to spread through the crowd that the bald man in the lumberjack shirt was Joe the Plumber. Within seconds he'd been mobbed. The panel discussion was part of the 2009 Conservative Political Action Conference, or CPAC, an annual gathering of rightwing activists, and soon news of Joe's arrival had filtered down the corridors to more of the 8,500 delegates. Young women bounded up with mobile phones, demanding photos. Young men in bow ties - there were a lot of young men in bow ties - thrust their hands forward. Joe posed obligingly for every photo, shook every hand.

"Oh, man, this is going to be my Facebook profile picture," one woman said to a friend. "You can fix my pipes anytime, Joe!" an older man yelled. Joe smiled, pretending to find the remark ingeniously amusing. True, he may not really be called Joe, and he may not really be a plumber, due to a few questions regarding his licence, and he may have come to symbolise the cheap populism and anti-intellectual rabble-rousing that helped lose John McCain the 2008 presidential election. But let it be stated for the record that Joe the Plumber is an incredibly patient man.

The clamour around him, though, was only one example of a strange jubilation I kept encountering at CPAC. Less than four months earlier, Barack Obama had won the election on a decidedly liberal platform; one month earlier, he'd been sworn in as president, and swiftly reversed several key Bush administration policies. Sarah Palin had returned to Alaska; McCain was lying low; media commentators were declaring the death of conservatism, and years of exile for the Republicans. Yet inside the Omni Shoreham hotel, America's conservative activists were in a state of frenetic excitement. Perhaps it was nostalgia for the campaign just past, or the thrill of being surrounded by like-minded people, but nobody seemed remotely depressed. Turning one corner, you'd find a group of student campaigners distributing Palin 2012 stickers and enormous full-colour posters of the Alaska governor; turning another, you'd find a crowd of thousands in the main arena cheering as John Bolton, the neoconservative, predicted Obama would be a one-term president, or going wild as Rush Limbaugh declared, "I want Obama to fail!"

In one room, the National Rifle Association was holding a raffle - imagine an English church fair, except with full-colour posters of semi-automatic weapons - while in another the Dutch politician Geert Wilders, recently banned from the UK because of an anti-Islam film he'd made, greeted an adulatory crowd. Stopping at a water cooler, I almost collided with the blogger Pam Atlas, who led the campaign to prove Obama had forged his birth certificate. She was delightedly hugging the author Jerome Corsi, whose book The Obama Nation, a hostile, error-filled attack on the Democratic candidate, had prompted a 40-page rebuttal from the Obama campaign.

It was hard not to see the merriment as taking place in a looking-glass world. Obama's victory had been not just decisive, but transformative, with dire consequences for the rightwing: the coalition of voters carefully assembled by Karl Rove - low-tax fiscal conservatives, the religious right and the tough-on-crime, anti-immigrant, law-and-order types - had been blown apart. Twenty per cent of self-described conservatives had voted for Obama, as had a quarter of evangelicals; according to some analyses, a majority of the electorate had considered Obama, not McCain, the low-tax candidate. Democratic canvassers reported incidents of racist voters - a reliably rightwing demographic if ever there was one - announcing their intention to vote for Obama because he was the more competent candidate, even though in the same breath they'd sometimes call him a nigger. The Republican party was in "a world of pain", the centrist Republican columnist David Brooks argued shortly after Obama's victory. "It's just a circular firing squad, with everybody attacking each other and no coherent belief system, no leaders. You've got half the party waiting for Sarah Palin to come rescue them. The other half waiting for Bobby Jindal, the Louisiana governor, to come rescue them. But no set of beliefs. Really a decayed conservative infrastructure."

In the wake of defeat, neither self-criticism nor introspection seemed to be on the agenda at CPAC. Virtually the only negative emotion I encountered came from Joe the Plumber. Plucked from obscurity after being filmed asking Obama a question about small business taxes, he'd recently co-written a book, Joe The Plumber: Fighting For The American Dream, and had been dispatched to Israel as a war correspondent for the conservative website Pajamas Media. But when, after the panel discussion, I asked if he was having fun, he thought for several seconds, then replied, "No. I'm really not."

Not at all? "No, man, no. I look upon this as a responsibility. It's something I gotta do: try to change things, make a difference. But, to be quite honest, I wish it had never happened." He looked out over the milling delegates, and sighed. "I never asked for any of this," he said.

A man with a clipboard approached, snapping Joe out of his reverie. "Fox News is ready for you now," he said. Joe the Plumber extended his hand towards me. "OK, brother, nice to meet you," he said, then ambled obligingly off.

What does it mean to call yourself a conservative in America, or anywhere, in 2009? The word, in its oldest sense, evokes a respect for tradition, a suspicion of radicalism and a preference for gradual, organic change by individuals rather than sweeping government projects. Yet the Republicans lost in 2008 after eight years of rule by radicals, whose conservative ideology included the grandest of government projects - reshaping the Middle East by force - along with a historically unprecedented centralisation of power. Very unconservatively, Bush implemented the largest expansion of health benefits in US history, then authorised the first multibillion-dollar bailout of Wall Street. To be sure, many people at CPAC, like conservatives I met elsewhere, felt betrayed by Bush. ("Karl Rove should crawl back to Texas, curl up beside a cactus and contemplate the ruin he has inflicted on the party," a panellist told delegates.) But they, too, had radical plans: to overthrow the legality of abortion, to close down America's borders, to leave the UN, to impose a flat tax, to overturn the consensus on global warming. And nobody could agree on who the "real" rightwingers were. Moderate Republicans blamed Palin's shrill attacks on "elitists" for destroying McCain's credibility; for those on the right of the party, choosing her was the only truly appealing thing the alarmingly liberal McCain ever did.

In British politics, the solution to such a muddle might be for a fresh-faced leader - a Blair or a Cameron - to take on his own party, forcing reform on the infighting members. But it's far harder for such a leader to emerge in the US if your party doesn't control the White House, harder still if you're a minority in Congress. "In Britain, political change is always imposed from the top down - half a dozen people who have houses next door to each other in London come to an understanding, win a contest and impose their vision on the party," says the former Bush speech-writer David Frum. "In America, change tends to come from the middle up - from the activists." And the problem with activists is that they tend to prefer passionate commitment to pragmatism. "I call them say-it-louder conservatives," says Frum, in a coffee shop around the corner from the American Enterprise Institute, a neoconservative thinktank where he now has an office. "If people don't like what you're saying, say it louder! Then they'll like it!"

It's odd that Frum has emerged as a figurehead of the moderate reformers - those calling for the American right to abandon its extremes and accept that issues from global warming to the financial crisis require a new approach. In Europe, he's probably best known as the person who coined the phrase "axis of evil" (though his version was "axis of hatred"). He co-authored a book with Richard Perle defending the Iraq war, and a book in praise of Bush, The Right Man. But now he says US conservatism has suffered a "psychic defeat" - a disease of which the Palin-worship at CPAC was a symptom. "One of the stages in the decline of a political movement is the moment when it comes to feel beaten. And in that moment it becomes reactionary, because there's a sense that to engage with the modern world in any way is to give up your beliefs."

A big part of the problem, Frum argues, is precisely that American conservatism was so successful. From this perspective, the popular revolution spearheaded by Ronald Reagan identified real problems - crime and chaos in American cities, rampant inflation, industries held hostage by unions - and implemented tough solutions, which worked. "The things we promised to do, we did." Government shrank; taxes were cut; worker protections were reduced: the free market, in short, was given a freer hand. Today's conservative activists were in their 20s then, excited by the sense of change. "But now 30 years have passed, and we're 30 years older, and what happens to the mind as it approaches 50 is it tends to become more rigid, more reluctant to absorb new impressions. We need to understand that the country has new problems, which require new solutions."

Cutting taxes, for example, might have been the right thing to do in 1980 - but if you keep on blindly cutting taxes, as if always cutting taxes were a keystone of conservatism, you'll reach a point where people think taxes are low enough; they don't want to lose the services the taxes pay for. "It's calcification," Frum says. "Our problem is we make principles of our policies. Limited government is a principle. The idea that income tax rates must always be cut, no matter what they are? That's not a principle." Frum's plan, outlined in his book Comeback: Conservatism That Can Win Again, calls for the Republican party to emphasise healthcare reform over tax cuts, to develop an environmental message, tone down the pro-life, anti-gay marriage rhetoric, and focus on the factor that drew so many conservatives away from McCain/Palin and towards Obama: not political ideology, but the sheer question of competence.

It goes without saying that this message doesn't go down well with many activists - especially as it comes from one of the pro-war neocons, whom they blame for Bush's slide into unpopularity. Tax cuts work, goes their counter-argument, whereas "conservative environmentalism" is just an untested idea. Commentators such as Frum, one prominent rightwing activist told me in disgust, "have never elected a dog-catcher, but they've got theories about how the rest of [us] should spend [our] resources and time and energy... They should be resigned to the fact that those of us who actually work all day will be making the decisions." He chuckled derisively. "There is no problem in America that requires more state power, more taxes, or more control over your life."

In North Carolina (which went for Obama although the state hadn't voted for a Democratic presidential candidate since Jimmy Carter), I meet Jeff Katz, until recently a flagship rightwing radio host, who blames Republicans' naivety with the internet for their election defeat. "People got lazy," he says. "And the party, not just here, was not really tied in to the way that modern campaigns had to be conducted." It was an argument I kept hearing. At the Conservatism 2.0 event, there was a palpable sense of resentment - as if the web, with its free market spirit, ought rightfully to have been the preserve of the right.

In the days and weeks following CPAC, the divisions on the US right became only more pronounced. The centrepiece was a public spat between conservative commentator and radio host Rush Limbaugh and the new chair of the Republican National Committee, Michael Steele, whose efforts to emulate the Obama campaign's path-breaking tactics had a cringeworthy, your-dad-at-a-disco quality. (What the party needed, he argued, was to apply its timeless conservative principles in "urban-suburban hip-hop settings".) Steele, radically for a Republican, called abortion "an individual choice"; he dismissed Limbaugh as an "entertainer", before recanting. White House operatives actively encouraged the perception that the divisive Limbaugh, with his 13m weekly listeners, was the unofficial leader of the party, too powerful for the official leaders to ignore.

As Democrats pressed ahead with Obama's stimulus plans, and his $3.5 trillion budget, Republicans flailed. Instead of calling for more modest expenditure, they proposed a spending freeze, attracting derision from economists. And they played Obama's stimulus for laughs, mocking, for example - in a speech by Jindal - an expenditure of $140m on volcano monitoring. (A month later, Alaska's Mount Redoubt erupted.) "The political philosophy of the GOP [Grand Old Party] right now seems to consist of snickering at stuff they think sounds funny," wrote Nobel prize-winning economist and New York Times columnist Paul Krugman. "The party of ideas has become the party of Beavis and Butt-head."

It is far too early to tell which of the competing visions for the Republican party will prevail. Much hangs on the success or failures of the Obama administration, and on the available choices to lead the party: to the horror of many moderates, Palin does seem to be in with a chance. But there's plenty of time for support to congeal round a less incendiary alternative: Jindal, an Indian American, to the right on all the key issues, whose early forays into national politics have been awkward but who might offer an Obama-esque mix of youth, competence and ethnic diversity; Mitt Romney, beaten to Republican nomination for president by McCain in 2008; stimulus-hating South Carolina governor Mark Sanford; or generally moderate Minnesota governor Tim Pawlenty. Such questions determine the future of a party that can no longer rely on the coalition constructed by Rove, playing off the conservative base against middle-ground votes.

What seems inevitable, though - and what even some of the reformists accept - is that the hardliners will dominate the party for the immediate future. Within days of the election defeat, you could see them moving into position. Key figures gathered in Virginia at the home of Brent Bozell III, founder of the Media Research Centre, a fierce opponent of abortion, "indecency" and the alleged distortions of the godless liberal media. (The MRC coordinated the protest at Janet Jackson's 2004 Super Bowl "wardrobe malfunction".) The meeting reached the conclusions you might have expected, given the attendees: that the defeat wasn't the movement's fault but McCain's, and it showed that, deep down, America was conservative.

Bozell is 53, bearded, jovial and rambunctious; he despises the American liberal media, but says he's happy to talk to journalists from leftwing British publications because they don't pretend to be neutral. His headquarters is filled with banks of televisions and dominated by vast group oil paintings of rightwing media heroes: Limbaugh, Rupert Murdoch and someone who appears to be the online gossip-monger Matt Drudge.

Bozell's argument is simple: America is conservative, but McCain wasn't; when it came to the economy, the public saw Obama as the careful, conservative candidate. (Polls do provide modest support for this thesis.) "We conservatives didn't play a role in this presidential campaign," Bozell says. "This was the most liberal Democrat in the Senate, versus the most liberal Republican running this year." Like the delegates at CPAC, he doesn't seem troubled by the outcome. On the contrary, he believes the 2008 election destroyed the credibility of McCain-style centrism, opening the gates for a resurgence of hardcore conservatism. "The Republicans have surrendered their brand. They're in serious, serious trouble." He swings his feet up on to a coffee table and leans back with his hands behind his head. "Conservatives can come away with great relief, and great optimism, that the American people are still on their side. There was this terrible butchery of the Republican party, and a terrible candidate, with a terrible message - versus something that came out of central casting for the Democrats! I mean, could you come up with someone better than a half-black, half-white person who sounds like Shakespeare and can convince the American public he believes what you believe in, no matter what you believe in?" Find a more telegenic candidate, a Palin or Romney, say, with a crystal-clear conservative message, to run in 2012 against a tired Obama, his honeymoon long over, and the next election is there for the winning.

The idea that the right should become more moderate, more accepting of a bigger government role in, say, healthcare and the economy, Bozell argues, is "foolish. I mean, what the hell is that? It's called liberalism! What else do you liberals care about? You believe government can do great things for the public! So when a conservative says it, how is that conservative? It's liberal! Fine, but don't call yourself a conservative! You might be a Rastafarian, for all I know, but you're not a conservative!"

The radical anti-tax campaigner Grover Norquist, another attendee at Bozell's summit, argues that what reformers forget is that voters will tell pollsters they support things - like an expanded role for government in healthcare - then backtrack speedily if asked to pay for them. In Norquist's view, American politics consists of two coalitions. One side, on the left, wants "to take things from people and give it to themselves". The other, on the right, may seem like a motley collection of gun-owners, home-schoolers, libertarians, religious devotees and anti-tax activists, but have one thing in common: "They want to be left alone." It is intriguing to speculate what effect Obama's economic rescue package may have on the leave-us-aloners. If it works, might they come to recognise the value of greater government activity? If it fails, will their numbers grow, as more Americans reject the notion of paying more in taxes to shore up the economy? (There were early signs of such resentment in the ecstatic response to a rant delivered by the CNBC journalist Rick Santelli, in which he attacked the idea of helping those unable to pay their mortgages, calling them "losers" who didn't deserve to be supported by those who'd managed their money better.)

The sense I found hard to shake at CPAC, though, was that most of those present weren't particularly interested in the story of economic catastrophe unfolding in the media and in hundreds of thousands of households. True, there were plenty of attacks on Obama's "European-style socialism". But mainly people were busy with their specialist interests: promoting the organisation Parents And Friends Of Ex-Gays, or Youth For Western Civilisation, fighting to eliminate the scourge of political correctness on America's university campuses, laughing at "global warmists", or cheering Wayne LaPierre, executive vice president of the National Rifle Association: "Our founding fathers understood that the guys with the guns make the rules!" Delegates wore NOBAMA T-shirts and shopped at stalls selling Obama Waffles. They were enjoying themselves, hunkering down together, declaring their difference from the grudging consensus that was coalescing, just down the road on Capitol Hill, behind Obama's vision for America.

I ran into Joe the Plumber one more time, at a book signing at the same moment that actor Stephen Baldwin, the rightwing brother of Alec, Daniel and William, strolled across to greet him. The two burly men shared an awkward hug.

"I've been prayin' for ya!" Baldwin told Joe.

Joe looked bemused. "What are you doing here?"

"I'm trying to get the conservative thing to be a little hipper here," Baldwin responded. He gestured to indicate the room, the conference, perhaps the conservative movement as a whole. "Because you guys could do with some help."