In the early 1980s veteran pollster Stan Greenberg, conducted a focus group in Macomb County, a Detroit suburb, of former Democrats who had switched allegiance to the Republican Ronald Reagan. After he read a statement by Robert Kennedy about racial inequality, one participant interjected: "No wonder they killed him."

"That stopped me and led to a whole new analysis of Reagan Democrats," wrote Greenberg in a recent report, Inside the GOP. "I realised that in trying to reach this group of people race is everything," he told me.

While conducting a focus group with Republicans over the summer he had a similar revelation, although it came not from a sole outburst but almost throwaway comments, often left on cards after the session. As one man left his handout he half-joked: "It's probably digital, so you can check it on the NSA files." Another asked: "Now you're going to guarantee that what we put down here, we won't be getting a call from the IRS about an audit or anything like that?" Alongside this sense of being spied upon was relief that, in these Republican-only groups, they had found kindred spirits. "I'm not alone in the way I view things for the most part," wrote one on a postcard. "Not by myself in thought process," confided another.

Those seeking to understand what drove the Republican party to shut down the government this month in a strategically disastrous move that laid bare its deep internal divisions – and ultimately led to humiliating defeat – could do worse than start here. The report reveals a sense of ideological, demographic and cultural siege, on the American right, from which there is no obvious escape. Unable to comprehend or process last year's election defeat, they feel the nation has become unmoored from its founding principles and is on a full-scale, unrelenting descent into chaos. Obama has been victorious in implementing socialism and the party they identify with has proved incapable of halting the decline, leaving them alienated not only from the country at large but one another. If it appears as though they are howling at the moon, it's because they feel all earthly options have been exhausted.

Describing Ireland's economic and cultural transformation in his book The Deportees, Roddy Doyle wrote: "I went to bed in one country and woke up in a different one." Many Republicans have precisely the same feeling.

Central to this deep-seated sense of angst is race. In 2012, 92% of the Republican vote came from white people who, within 30 years, will no longer be in the majority. "They are acutely racially conscious," says Greenberg. "They are very aware that they are 'white' in a country that is becoming increasingly 'minority'." Growing increasingly dependent on an ever-shrinking base, they see their electoral fortunes waning but are resistant to adapting their message to broaden their appeal beyond their narrow racial confines. Race is less the explicit target of their anxiety (issues such as affirmative action and civil rights no longer dominate) than the primary (if not exclusive) prism through which their political consciousness is being filtered. "Race," writes Greenberg, "is central to their worldview."

There are three main ways in which this has been a factor in the recent government shutdown and Republican schisms. First is gerrymandering. Since race is one of the best predictors of voting behaviour, House congressional seats have been manipulated largely on racial grounds. Politicians at state level carve constituencies into odd and unlikely shapes, shuffling around various racial groups to protect incumbents. Both parties do this when they have the chance but Republicans, who run more state houses, have had more chance and have undertaken the task with much zeal and guile. As a result, in 2012 the Democrats won more votes nationally for Congress but still got fewer seats, giving the Republicans who shut down the government a fragile mandate. It also means incumbents need not fear losing their seats, leaving them able to act out.

Second is the perceived beneficiaries of government spending. Republicans are more likely to regard intervention as being to support minorities rather than to support the poor. This goes not only for food stamps and welfare but also for Obamacare – which was the issue that initially sparked the shutdown.

"Obamacare is a racial flashpoint for many evangelical and Tea Party voters," writes Greenberg. Their despair is largely rooted in the assumption that by championing programmes that disproportionately help minorities, Obama is effectively buying votes and securing a growing tranche of the electorate who will for evermore be dependent on government. One participant, echoing the views of many, said: "Every minority group wants to say they have the right to something, and they don't. It's life, liberty and the pursuit of happiness. It doesn't say happiness. You get to be alive and you get to be free. The rest of it's just a pursuit … you're not guaranteed happiness. You have to work for it."

Finally, there is Obama – the black son of an African immigrant and white mother – who stands as an emblem for all this unease, personifying, in their minds, not only their political impotence today but their demographic irrelevance tomorrow. The word they're most likely to use to describe him is "liar". But their hostility goes beyond his policies and pronouncements to a deeply rooted suspicion of his authenticity.

"[There] is a sense of him being foreign, non-Christian, Muslim – and they wonder what really are his motives for the changes he is advancing." As he moves into his second term, there is now an elision in the Republican mind between what they think he is (an immigrant, a fraudster, a non-American) and what they think he does (assist immigrants and fraudsters in contravention of American ideals).

Their inability to craft a credible strategic response to these insecurities only serves to reinforce them. "You don't like a particular policy or a particular president?" taunted Obama last week. "Go out there and win an election." The trouble is Republicans can't because their racially charged rhetoric alienates minorities, leaving them more electorally isolated, prompting defeat – which leaves them ever more divided. Meanwhile, their reckless obstruction in Congress, which nearly triggered a default, makes the nation's descent into chaos more likely. Unable to come to terms with the country in which they live, they are complicit in creating the very future they most fear.

Twitter: @garyyounge