In the Middle Ages, Italian city-states led the European “commercial revolution” with innovations in finance, trade, and technology. Then something strange happened. In 1264, to take one example, the people of Ferrara decreed that, “The magnificent and illustrious Lord Obizzo … is to be Governor and Ruler and General and permanent Lord of the City.” Suddenly, a democratic republic had voted itself out of existence.

In fact, this was not an uncommon occurrence in Northern Italy at the time. As Niccolo Machiavelli explains in The Prince, the people, seeing that they cannot resist the nobility, give their support to one man, in order to be defended by his authority. The lesson is that people will abandon democracy if they are worried that an elite has captured its institutions.

Medieval Italy’s democratic institutions succumbed to what we might now call populism: an anti-elitist, anti-pluralistic, and exclusionary strategy for building a coalition of the discontented. The method is exclusionary because it relies on a specific definition of “the people,” whose interests must be defended against not just elites, but all others. Hence, in the United Kingdom, the Brexit leader Nigel Farage promised that a vote for “Leave” in 2016 would be a victory for the “real people.” As Donald J. Trump told a campaign rally the same year, “the other people don’t mean anything.” Likewise, former Colombian President Álvaro Uribe often speaks of the “gente de bien” (the “good people”).

There are two obvious reasons why such populism is bad. First, its anti-pluralistic and exclusionary elements undermine basic democratic institutions and rights; second, it favours an excessive concentration of political power and de-institutionalization, leading to poor provision of public goods and subpar economic performance.

Nonetheless, populism can become an attractive political strategy when three conditions obtain. First, claims about elite dominance must be plausible enough that people believe them. Second, in order for people to support radical alternatives, existing institutions need to have lost their legitimacy or failed to cope with some new challenge. And third, a populist strategy must seem feasible, despite its exclusionary nature.

All three conditions can be found in today’s world. The increase in inequality over the past 30 years means that economic growth has disproportionately benefited a small elite. But the problem is not just inequality of income and wealth: there is also a growing suspicion that the social distance between the elite and everyone else has widened.

These economic and social disparities have profound implications for political representation. In the US, political scientist Larry M. Bartels has shown that while legislators have increasingly defended the interests of the rich, gerrymandering has spared them from political competition. In Europe, Jean-Claude Juncker, while serving as prime minister of Luxembourg, once described the European Council’s decision-making as follows: “We decree something, then float it and wait some time to see what happens. If no clamour occurs … because most people do not grasp what had been decided, we continue – step by step, until the point of no return is reached.” Such elitist logic is intrinsically vulnerable to populism.

As a de-institutionalising strategy, populism appeals to the growing cohort of those who are disillusioned with existing arrangements. In the US, the widespread perception that institutions have failed to address issues such as inequality has been eroding public trust in major institutions since the 1970s.

After failing to anticipate the 2008 financial crisis, US policymakers are now struggling to regulate (and tax) new “mega-firms” like Amazon and Facebook. They are also seen as having dropped the ball with respect to globalization and the effects of the “China Shock” on local labour markets. Similarly, in Europe, increased labour mobility and rolling refugee crises are widely seen as having surpassed EU institutions’ carrying capacity.

In addition to managing new challenges poorly, institutions and policymakers have also failed to look beyond their own dominant narratives. For example, in the run-up to the Brexit referendum, the “Remain” campaign focused entirely on the economic costs of leaving the European Union, even though opinion polls showed that migration and other issues were of much greater concern to voters.

Finally, for populism to get a foothold, politicians themselves must see it as a viable strategy. Generally speaking, declaring that the “other people don’t mean anything” isn’t the best way to garner broad support. So, even when structural factors favour it, populism can succeed only in certain circumstances. In Trump’s case, the intense partisan polarisation in the US means that he can appeal to marginal or swing voters because he knows that Republicans will vote for him no matter what. And, more generally, populism can win when the “other people” are narrowly defined or simply small in number, provided that they can still be depicted as posing a threat.

To defeat populism, then, one must address all the factors that make it a viable strategy. That starts with recognizing that populism can emerge only when there are real social and economic problems to give it electoral traction. It also means being honest about the fact that there are competing and contested visions of citizenship, which should be debated, not ignored.

Finally, we need more democracy and representation – including, possibly, referenda – so that voters feel as though their concerns are being taken seriously. The political class should be exploring new ways to make government more representative of society. India, for example, has caste-based quotas for parliamentary seats and other positions, and many other countries do the same with respect to gender. There is no reason why the US and Europe couldn’t pursue similar measures.