One of America’s defining characteristics used to be optimism. That’s not just a stereotype or a cliché. It has been captured by a number of opinion polls. A 2015 Pew Research survey, for instance, found that Americans were much more likely than citizens of other rich nations to describe their day as “particularly good”. They tend to view their country in positive terms too: an earlier Pew survey found that Americans were more likely than citizens of other countries to describe their culture as superior. And a 2017 study for the US’s National Bureau of Economic Research found that Americans tend to be more optimistic about prospects for social mobility than Europeans and were more likely to consider their society fair.

America’s national identity is built on the dream of individual aspiration and self-improvement, but for a long time this has existed more as a kind of mental state than a real feature of American society. Actual social mobility in the US is low and declining and inequality is increasing. For all the national myth-making, it’s better to be born poor in Denmark, or any other Nordic country, than to be born poor in the US.

An optimistic disposition also hasn’t made Americans happy. One eighth of the adult population is on antidepressants and reported rates of mental illness are rising. Last year, the US ranked 18th in the United Nations world happiness survey, having dropped four places since 2017. It ranks behind the Scandinavian countries, Australasia and several northern European countries – though not the UK, which ranks 19th.

In her 2009 book Bright-Sided, a powerful treatise against positive thinking,Barbara Ehrenreich grappled with the apparent contradiction between American positivity and deep levels of inequality and unhappiness. “The answer, I think, is that positivity is not so much our condition or our mood as it is a part of our ideology – the way we explain the world and think we ought to function within it,” she wrote. There’s an anxiety, Ehrenreich argues, at the heart of this national cult of positive thinking, a suppressed knowledge that in reality things aren’t that great: “If the arc of the universe tends toward happiness and abundance, then why bother with the mental effort of positive thinking? Obviously, because we do not fully believe that things will get better on their own.”

Interestingly, however, this now appears to be changing. The Trump administration precipitated a national crisis of confidence and a new era of American gloominess. Liberals and anti-Trump conservatives are dismayed by the president’s open misogyny and racism, by the horrors of family separations, and by the long-term consequences of Donald Trump’s disregard for the truth and his disdain for American democratic norms and institutions. Trump supporters aren’t exactly upbeat either. His success as a politician has relied on his knack for stoking the politics of fear, anger and racial resentment. These days, one of the few things uniting the right and left in American politics might be the belief that America is on the brink of disaster.

****

American presidents are usually optimists. At turn of the millennium, Bill Clinton boasted in his state of the union address that “never before has our nation enjoyed, at once, so much prosperity and social progress with so little internal crisis and so few external threats”. When George W Bush announced his candidacy, he declared, that: “I’ve learned that people want to follow an optimist… They respond to someone who sees better times, and I see better times.” Bush was so dangerously inclined towards optimism that some of his staff reportedly learned not to express hesitation or doubt in front of him. Barack Obama, whose political manifesto was titled The Audacity of Hope, is the archetypal optimist. In his final address to the nation, ten days before Trump assumed office, he said that he was leaving “this stage tonight even more optimistic about this country than I was when we started”.

Trump is different. His inauguration speech, with its ominous description of “American carnage”, was much darker than those of his predecessors and though ostensibly his message is one of national renewal his tone was grim, and at times menacing. “At the bedrock of our politics will be total allegiance to the United States of America,” he said, sounding more like a headmaster delivering a punishment than a new leader seeking to inspire the nation.

Trump does not know how to harness emotions such as hope or optimism. That’s why he stoked fears of a migrant “invasion” during the 2018 midterms and instigated the longest government shutdown in US history over his demand for a $5.7bn border wall, a trumped-up solution to a trumped-up crisis. His political stance seems instinctive: he is a man who smiles rarely and never laughs, who seems primarily driven by greed, personal ambition and resentment. For the more cerebral members of the Trump administration, political pessimism is an ideological choice. Trump’s former chief strategist, Steve Bannon, is convinced that America is facing an existential crisis that can only be averted through a populist revolt and the destruction of the state.

The shock and trepidation caused by the Trump era has prompted a sombre re-examination of America’s recent history, from the rise in far-right extremism that went largely unnoticed during Obama’s time in office, to the shifts in the American economy that have stalled social mobility and decimated job opportunities in large parts of the country’s interior.

****

The opioid epidemic is perhaps the most damning indicator that something is seriously wrong in America. It is a public health crisis that, not coincidentally, has struck hardest in the largely white, rural parts of America that voted for Trump. More people are dying of drug overdoses in America per year than ever died of Aids at the height of the epidemic. As a result, US life expectancy (which already trailed behind western Europe, Australasia, Japan and South Korea) has fallen for three consecutive years, something that just isn’t supposed to happen in rich, developed countries. The opioid epidemic was spurred not only by America’s unequal health care system and its rapacious pharmaceutical industry, but also by despair and desperation among the communities most affected by post-industrial decline. As the journalist Andrew Sullivan wrote in a masterful essay for New York Magazine, it is “a story of how the most ancient painkiller known to humanity has emerged to numb the agonies of the world’s most highly evolved liberal democracy”.

The Guardian journalist Sarah Smarsh’s Heartland (Scribe), published last year, is one of several recent white, working-class memoirs that seeks to make visible an aspect of American life that has been overlooked by the coastal elites for decades. Heartland is a loving history of Smarsh’s family, who have been trapped in poverty in rural Kansas for generations. Her relatives are hard-working and eschew government help on moral principle, but their odds of achieving financial comfort are tiny and dwindling, thanks to the rising financial pressures on family farms, the lack of affordable health care, stagnant wages and de-funded public schools.

American families such as Smarsh’s – white, working class and poor – are “wilfully ignored in the modern story of our country”, she observes. “The defining feeling of my childhood was that of being told there wasn’t a problem when I knew damn well there was.” The national myth is that America is a classless society, a land of abundance and opportunity, where if you can’t improve yourself you are the problem. This myth is so powerful and enduring that her relatives believe it too – even as they face disadvantages too great to overcome with a positive disposition and a good work ethic.

Heartland takes a very different approach to Hillbilly Elegy (William Collins), a memoir of growing up poor in Appalachia by the conservative writer and venture capitalist JD Vance. Hillbilly Elegy was published in 2016 and became a New York Times best-seller as American pundits scrambled to understand how Trump’s election took them all by the surprise. Although job opportunities have been declining in Appalachia for decades, Vance is less interested in economics than in culture. He is concerned that the people he grew up with are “reacting to bad circumstances in the worst way possible” and that their culture “encourages social decay instead of countering it”. As he sees it, too few people in his home neighbourhood are willing to work, too many are addicted to drugs or alcohol, too few are able to make a marriage work or provide stability, care and spiritual guidance to their children.

While Vance places responsibility on the individual, Smarsh is more attuned to their circumstances. Her father, for instance, struggled with alcoholism only after he almost died of chemical poisoning at work and plunged into depression. The left and right might not agree on solutions, but since Trump’s election at least they both agree that these problems are too big to ignore.

****



Last post: a Janesville resident shows support as GM workers end their final shift, December 2008​

In addition to the proliferation of working-class memoirs, the past few years of American soul-searching have given rise to a spate of insightful, deeply reported books that ostensibly chronicle America’s decline but also offer rich insights into the meaning and limits of American optimism. One of them is Janesville (Simon & Schuster), a 2017 book by the Pulitzer-Prize winning Washington Post reporter Amy Goldstein, who spent five years following the stories of residents of Janesville, a town of 63,000 people in Wisconsin, after General Motors shut down its assembly plant in 2008. Some 3,000 people lost their jobs overnight, devastating the local economy, while in the aftermath thousands more lost their jobs.

The residents did not give up, though. Instead, they showed extraordinary resilience and determination to lift up their community and find new work. Local politicians and business leaders rallied to put together the biggest incentive package in Wisconsin history for GM to return to Janesville. They lost their bid. It turned out they never stood a chance of winning. The head of the Janesville job centre created a coalition of local organisations tasked with helping the newly unemployed retrain and find work. He attracted millions of dollars in outside grants and many laid-off workers, determined to turn their misfortune around, signed up for training at their local college.

As the years stretched on something became disturbingly apparent. Most people who found new work had to accept lower pay and fewer benefits. The optimists, those who had tried hardest to work themselves out of their rut, tended to fare worse. Of the 2,000 laid-off workers who retrained, only a third later found a steady job. Among those who did not retrain, half found steady work. Those who retrained also tended to be earning less and the longer they trained, the less they eventually earned.

Kristi Beyer, the star of the training centre, a hard-working, straight-A student who earned an associate’s degree in criminal justice and retrained to become a prison guard, struggled under the emotional strain of her new job. Her marriage suffered, because she and her husband worked opposite shifts, and she started an affair with a prisoner. Soon after the affair was discovered, Kristi went shopping with her mother and was delighted to buy a pair of new size-eight jeans after a successful diet. She got a new tattoo, in honour of her son and her new boyfriend, and then she went home and killed herself.

Janesville is an extremely moving portrait of individuals who embody many of the qualities most celebrated in American political culture: entrepreneurialism, grit, generosity, community-mindedness, familial loyalty and, of course, optimism. But it also demonstrates the dangers of false hope. The people of Janesville do all they can to improve their lives, but they are the victims of economic and political decisions far beyond their control. Under these circumstances, what is the use of American optimism?

****

The Harvard sociologist Matthew Desmond won the 2017 Pulitzer Prize for his book Evicted (Penguin), another ambitious, deeply researched portrait of American decline. Desmond draws on years of embedded fieldwork to tell the stories of eight families teetering on the edge of homelessness in Milwaukee’s low-quality private housing market, and the landlords who are turning a profit renting to the very poor. Eviction has become an overlooked but everyday aspect of American poverty, with millions cast out from their homes each year. Rents, even for unsafe, unsanitary housing, are soaring and because of high demand at the bottom of the market, landlords are incentivised to turf out tenants who don’t pay on time, or put in maintenance requests, or call 911 too often.

High eviction rates have consequences: crime rates soar in previously safe neighbourhoods as residents become more transient and no longer watch out for one another. But the most devastating impact is on the people evicted: women forced to choose between an abusive relationship or homelessness, mothers who cannot provide for their children, struggling families who cannot begin to improve their lives while all their money and energy is focused on keeping off the streets.

Desmond follows the story of Sherrena Tarver, a former school teacher now making a fortune as an inner-city landlord. Sherrena would no doubt see herself as embodying certain American ideals: she is fearless, a go-getter, a skilful and self-made businesswoman who has carved out a niche for herself renting property in areas other landlords don’t dare to venture. She is not prone to crises of conscience when she evicts adults and children with nowhere else to go – because she has bills to pay too, as she often reminds her tenants. Her hard-won American dream generates nightmares for her tenants. When a fire breaks out in one her flats, killing a baby, Sherrena checks with the fire department if she has to return the parents’ rent given that the fire broke out at the start of the month. She’s pleased to learn she doesn’t.

Desmond’s book illustrates that something has gone horribly wrong with the American idea of self-improvement if it can only be achieved through trapping others in misery. “America is supposed to be a place where you can better yourself, your family, and your community. But this is only possible if you have a stable home,” he writes.

****

Perhaps the only optimistic book about America’s heartlands to emerge in the Trump era is Our Towns (Pisces Books), a national bestseller by James Fallows, a correspondent for American magazine the Atlantic, and his wife Deborah Fallows, published last year. The couple traversed the country by plane (he is a pilot) visiting small towns and speaking to residents and local politicians about almost everything except national politics. While they accept that the US is experiencing serious problems, they argue that many small communities are nevertheless flourishing. They point to a resurgence of local politics, downtown regeneration, improvements to schools and expansions to libraries, and the number of young professionals who are leaving the coastal cities to return to the small towns they grew up in.

The Fallows blame the TV media for contributing to unwarranted levels of pessimism about the state of the US. “What 24-hour cable news introduced and Fox perfected in the modern news consciousness is an unending stream of horrors from… somewhere else. The natural result of well-meaning liberal media is thus a kind of pity for the heartland, and of conservative media, a survivalist fear about what people Out There are trying to get away with,” James Fallows writes in the Atlantic.

Pity or fear, like despair or hopelessness, are not helpful responses to America’s predicament. Democratic victories in the midterms, and the newly youthful, diverse and female Congress, are raising hopes for a way out of Trump-era turmoil. But the end of American optimism could be a good thing. Faith in the American dream has persisted for too long, even as social mobility has stalled, jobs have disappeared and the population has sickened. If you believe that American society is fair, that this is a land of equal opportunity and abundance, then the logical conclusion is that the poor are at fault. A less optimistic America could be a more compassionate country, one more willing to expand the social safety net, treat addiction as an illness and expand health care provision for the poor. It could be a more prosperous country, one willing to set aside toxic partisanship and address the structural problems that are holding too many Americans back. It could even be a happier country, one in which poverty or worklessness is not seen as a moral failing. One can only hope, after all.

Sophie McBain is North America correspondent for the New Statesman