Mitt Romney has a John Kerry problem. And I don't mean being a wavy-haired, wealthy francophone from Massachusetts (although, in this climate, that's burden enough for a Republican nominee). He's been framed. And so long as his opponent's attacks sufficiently fit this frame, then they do not need to be accurate, honest or reasonable. They just have to fit.

For Kerry in 2004, the problem was inauthenticity at a time of war. Republicans successfully cast him as a man of great electoral opportunism and few principled convictions. After that charge had stuck, pretty much anything was fair game. His windsurfing, hair mousse, propensity for polling, position on the war and Vietnam record were all essentially similar codes for an identical accusation: this man will not lead; he will say what he thinks you want to hear. What you see is not what you get.

Whether this was fair or not is another matter. The point here is that it worked.

For Romney, the problem is the perception of a robber baron at the helm at a time of recession. His opponents have successfully framed him as an out-of-touch magnate with a tin ear for the travails of the common person. Most of the Democratic attacks of the last few months, over firings and outsourcings at Bain Capital, demands for his tax records, or about his wealthy donors, fit that mould. This rich guy doesn't understand you or what you're going through, and now he wants to buy the election.

And the evidence seems to be that it's working.

To compare this to "swiftboating" is inaccurate. The swiftboaters traded in clear inaccuracies, relied on a rightwing echo chamber and – while doing the GOP's bidding – kept at least the appearance of distance from George W Bush. The attacks related to Bain originated in the Republican party, now come straight from the president and have a basis in truth (although how relevant the truths are is debatable).

But there is this similarity between Kerry's and Romney's predicament. In both cases, his opponent's strategy of going negative gained traction not because of the veracity of the particular accusations, but because they confirmed a perception already in play about the candidate's character, a perception that had some basis in reality.

Put bluntly, Romney had a pre-existing condition. The image of him being effete was already out there. It was an issue for him during the Republican primaries, where he routinely lost among non-college white men who earn less than $50,000 a year – a key GOP demographic. As far back as 2008, Mike Huckabee observed:

"People would rather elect a president who reminds them of the guy they work with, not the guy who laid them off."

This is partly about how Romney made his money. When the rightwing pollster Frank Luntz addressed the Republican Governors Association in December, he told them:

"The public still prefers capitalism to socialism, but they think capitalism is immoral. And if we're seen as defenders of 'Wall Street', we've got a problem."

The form of capitalism Romney was involved in – venture capital for a private equity firm – has a parasitical ring to it that many find difficult to relate to. During one focus group of Republicans in Tampa, Ron Romonchuk, a retired consultant, called Romney "a 1%er" who has never gotten his hands dirty.

But it's also partly about how he performs his wealth. George W Bush came across as "regular" even if he wasn't. Romney truly hasn't got a clue even how to fake it. He went to Nascar to relate to working-class whites and told them he didn't follow the sport but knew a few of the owners. He went to Detroit and told a crowd he loved Cadillacs, which was why he had a couple. In the same focus group, deputy police officer Jonathon Ross said:

"It's hard to be around a guy with that much money."

Not all framings work. Hillary Clinton's chief strategist, Mark Penn, argued that Obama should be undermined on grounds of his cosmopolitan past.

"His roots to basic American values and culture are at best limited," Penn said. "Let's explicitly own 'American' in our programmes, the speeches and the values. He doesn't."

But attempts to frame Obama as "foreign", both by Democrats and Republicans, ultimately failed because they were too easily exposed as a proxy for "black", and in his manner and style there was nothing "foreign" about Obama apart from his name. And in a country with senators named Inhofe, Akaka, Murkowski and Barrasso, that didn't really mean much.

Nonetheless, however baseless, these attacks did speak to a broader matter that Obama was obliged to address. It is not difficult to see how things might have gone differently for Obama if he had not made his Philadelphia speech about race in America, in general, and his pastor, Jeremiah Wright, in particular. Rightly or wrongly (OK, wrongly), race was an issue he had to deal with swiftly and decisively or the charge of "foreignness" would likely have stuck. And in the absence of a firm rebuttal, all you can do, as Kerry did and Romney is now doing, is whine.

Where Romney is concerned, the precise details of when he ran Bain and why he won't disclose his tax returns are becoming irrelevant. He's looks like a very rich man who doesn't want to open his books – and that fits the robber baron frame.

"Perception is becoming Romney's reality, and these issues have now risen above mere distractions," says GOP consultant John Weaver, a senior adviser for McCain's 2000 and 2008 bids, talking to the Washington Post:

"The president has had the worst three months of any incumbent, due to the economy, since George HW Bush in 1992, and yet Romney has lost traction among key demographic groups in the vital swing states. He has got to get this behind him or he's going to face summer definition à la [Bob] Dole and [John] Kerry."

Romney's lawyerly explanations for not releasing his taxes, and keeping his name on Bain's filings only to retire "retroactively", may be sound. But as the saying goes, and John Kerry can testify, in American politics, "if you're explaining, then you're losing."

Follow Gary Younge on Twitter @garyyounge