In a surprising new national survey, members of each major American political party were asked what they imagined to be the beliefs held by members of the other. The survey asked Democrats: “How many Republicans believe that racism is still a problem in America today?” Democrats guessed 50%. It’s actually 79%. The survey asked Republicans how many Democrats believe “most police are bad people”. Republicans estimated half; it’s really 15%.

The survey, published by the thinktank More in Common as part of its Hidden Tribes of America project, was based on a sample of more than 2,000 people. One of the study’s findings: the wilder a person’s guess as to what the other party is thinking, the more likely they are to also personally disparage members of the opposite party as mean, selfish or bad. Not only do the two parties diverge on a great many issues, they also disagree on what they disagree on.

This much we might guess. But what’s startling is the further finding that higher education does not improve a person’s perceptions – and sometimes even hurts it. In their survey answers, highly educated Republicans were no more accurate in their ideas about Democratic opinion than poorly educated Republicans. For Democrats, the education effect was even worse: the more educated a Democrat is, according to the study, the less he or she understands the Republican worldview.

“This effect,” the report says, “is so strong that Democrats without a high school diploma are three times more accurate than those with a postgraduate degree.” And the more politically engaged a person is, the greater the distortion.

What could be going on? Bubble-ism, the report suggests. Even more than their Republican counterparts, highly educated Democrats tend to live in exclusively Democratic enclaves. The more they report “almost all my friends hold the same political views”, the worse their guesses on what Republicans think.

So do they believe in sticking with their own? No. When asked in a Pew survey whether it’s important to live in a place “where most people share my political views”, half of conservatives and only a third of liberals agreed. Although in principle more tolerant of political diversity, highly educated – and mostly urban – Democrats live, ironically, with less of it.

Something besides city living may be going on. Particularly among highly engaged Democrats, I think many are in grief. The recent losses have been hard to absorb: the presidency. The supreme court. The Senate. A majority of governorships. And of states in which the governor and both houses of the state legislatures are controlled by the same party, 22 are Republican and 14 Democratic. Withdrawal from the Paris agreement. Neil Gorsuch. Voter suppression. Children in cages. The future of the party highly unclear. Democrats are now the ones who feel like strangers in their own land.

In the face of loss or personal crisis – whether a pink slip, a medical issue or political defeat – most of us turn inward toward what feels self-confirming, comforting and safe. During the Great Depression, and more recently in the aftermath of the 2008 crash, those who had lost homes and jobs tended to volunteer less, visit less and generally retreat from the world, sometimes thereby making a bad situation worse. Today’s wounded warriors of the left – including many women and voters of color who have struggled mightily and fear their progress is in danger of going up in smoke – may feel a similar need for the familiar.

Being home surely restores one’s spirits. But in the long run, staying home leads to trouble. It keeps the left from knowing what it needs to know, good news and bad. It further marginalizes it. After all, in the heat of battle, the most brilliant generals are not those who insult the enemy from inside their own tent. It is those who most astutely gauge the troop count, the terrain and quirks of the opposing general.

In peacetime, too, in this political moment, the best approach is to learn about what is sometimes the alternative truth in which Fox News watchers live, the class and cultural grievances it appeals to and amplifies, and to understand, as one keeps listening, a curiously hidden moderation in substance and tone.

For starters, Democrats can recognize pieces of common ground where they already exist. For my 2016 book, Strangers in Their Own Land: Anger and Mourning on the American Right, I studied enthusiastic backers of Donald Trump in Louisiana – the last place on earth a Democrat might look for common ground.

But one man I talked with – someone raised on a sugar plantation, retired from a life-long career in oil, a proud member of the Louisiana Tea Party and a Trump supporter – grinned broadly at the mention of Bernie Sanders. “Free college? Free medical care? How yawl going to pay for that? He’s a pie in the sky guy,” he said. “But he’s a good man, Uncle Bernie.” Although an oil worker, he was a fan of clean energy, and liked the idea of a Manhattan Project to implement it.

Among Republicans, he isn’t alone. Despite the president’s denial of the climate crisis, national polls recently conducted by researchers at Yale, Stanford and Monmouth Universities show that a majority of voters in both parties now agree on many actions to mitigate it.

Should the US “set strict carbon dioxide emission limits on existing coal-fired power plants to reduce global warming and improve public heath”, Yale asked, even if “the cost of electricity to consumers and companies would likely increase”? Eighty-seven per cent of Democrats and 56% of Republicans answered yes.

“Should the US participate in the Paris climate accord and reduce greenhouse gas emissions regardless of what other countries do?” A majority of voters in both parties said yes.

A coalition of college Republican clubs recently endorsed a tax on carbon pollution.

There are other promising signs of middle ground. In response to the violence and death in Charlottesville, Virginia, big donors to the feuding parties, George Soros and David Koch, jointly funded the After Charlottesville Project to curb online calls to violence. The New York congresswoman Alexandria Ocasio-Cortez and the Texas senator Ted Cruz have co-sponsored a congressional bill preventing lawmakers from entering lucrative second careers as corporate lobbyists. Conservatives and liberals united to push for ex-felons in Florida to win the right to vote in 2018, and, this year to push modest but significant reform of our draconian criminal justice laws through Congress.

Such cross-party diplomats have not misjudged their opposite number – such a good lesson for us all.

When Martin Luther King marched from Selma to Montgomery, his fellow marchers included a priest, a nun, two rabbis, a Greek Orthodox archbishop and clergy of many Protestant denominations, black and white. In the long run, major political change virtually always requires rising up to cross a bridge – an urgently needed model for the Democratic party itself today.