With every new year, I typically set aside some time to write down what I’m grateful for. Health, family, friends, books, jazz, my dog, among other things. This year I added something I’ve been taking for granted. It’s democracy.

Like many of us, I have worried about the rising tide of rightwing populism, nationalism and polarisation across the world. Within just a few years, we’ve witnessed the election of Donald Trump in the US, the Brexit decision in the UK, the rise of Matteo Salvini in Italy, Victor Orbán in Hungary, the Freedom party in Austria and the Law and Justice party in Poland. The world’s largest democracy, India, is menaced by a newly virulent nationalism and xenophobia.

For a long time I wondered what explained the appeal of these apparently fringe movements that, in my view, had accidentally gone mainstream. They seemed like the exception to a general rule of progression towards, not away from, democratic norms. But this year I came to a different conclusion: it’s democracy that is a precious exception to the rule, and one that is extremely fragile, for a simple reason: the human craving for order and security when chaos feels imminent.

The philosopher and psychologist Erich Fromm first identified this predicament in his 1941 book, Escape from Freedom. The gist of it is this: when people perceive an increase in disorder, they feel tremendous anxiety. Inevitably, this anxiety leads to a quest for security. To bring a sense of safety back into their lives, they latch on to authoritarianism and conformity. As Fromm noted, this often leads to “a readiness to accept any ideology and any leader if only he offers a political structure and symbols which allegedly give meaning and order to an individual’s life”. He had observed this in Germany, which he fled in 1933: “Modern man still is anxious and tempted to surrender his freedom to dictators of all kinds,” he wrote.

Fromm was speculating about this dynamic. But decades later, I and other psychologists have empirically shown how insecurity is linked to the rise of autocrats and the erosion of democracy. In a survey we conducted before the 2016 US presidential election, for example, we asked US residents questions about how fearful they were about various threats, such as illegal immigration, a lack of jobs, crime, terrorism, an attack from Iran, among others. They also responded to statements aimed at gauging their desire for stricter rules and their support for different political candidates, including Trump. We conducted the same survey in 2017 in France, measuring support for Marine Le Pen.

The results of both studies were telling: people who felt threatened wanted to tighten up – to have stricter rules – which predicted their support for Trump or Le Pen in the US and France, respectively. Other research confirms the same pattern. Economic threats and the growing gap between the rich and poor also create a sense of chaos and instability. This has led to increased support for strong leaders willing to challenge democratic values and practices.

It’s a simple principle, one that is causing democracies all around the world to unravel. When people experience threat – whether actual or imagined – they begin to “tighten”. In physical terms, they tense their muscles, ready to defend themselves. In political terms, they begin to crave security and order in a community that seems to be collapsing. Authoritarian leaders satiate this need by promising quick, simple solutions – and, above all, a return to the tighter social order of yesteryear.

Leaders are aware of this basic psychology and exaggerate threats to gain popularity. Trump did so masterfully: at campaign rallies throughout 2015 and 2016, he warned his ever-growing crowds that the US was a nation on the “brink of disaster”. He cited Mexicans supposedly bringing violence across the border, global trade agreements and immigrants taking away jobs, and radicalised Muslims plotting terror on American soil. Throughout his campaign, he sent the clear message that he was capable of restoring social order: “I alone can fix it.” Analysing campaign speeches, we found that Trump used far more threatening language than Hillary Clinton.

To strengthen democracy, we’re going to have to deal with this threat psychology. Some of the threats, such as the loss of well-paid, secure employment, are real. We need to empathise with those who are struggling – rather than dismissing their fears – and develop innovative solutions, particularly for those put out of work by the decline of manufacturing amid the AI revolution.

But other threats are exaggerated and, unfortunately, they produce the same tightening and hostility. For example, in our studies, we’ve found that Americans greatly overestimate the percentage of people who immigrated illegally. (Republicans estimated that 18% of the US population is made up of people who are here illegally, while Democrats estimated that statistic to be less than 13%, on average. The actual figure, according to a 2017 Pew Research study, is closer to 3%.) The greater the misperception, the more people said they would vote for Trump in 2020. Ironically, many real threats – including violence and disease – have declined precipitously over the years, but manufactured or imaginary threats still persist.

Now more than ever, we need to develop ways to counter the misperceptions that are beginning to upend democracies. We can’t take those fragile political arrangements for granted. It is not primarily a question of weeding out particular personalities or reversing regrettable decisions, but addressing the things that made them attractive in the first place: that pervasive sense of rising threat.

• Michele Gelfand is a distinguished professor of psychology at the University of Maryland and the author of Rule Makers, Rule Breakers: How Tight and Loose Cultures Wire Our World