Last summer I attended a closed-door gathering of political scientists in Segovia, Spain, surrounded by the medieval spires of the Castile and León region. The scholars had flown in from across the world to spend a few days reviewing academic literature on the vexed subject of “populism”.

One of the academics showed me a graph charting how the number of Google searches for “populism” had rocketed in 2016, around the time of the UK’s Brexit vote and the election of US president Donald Trump, and remained high ever since. For decades a niche topic mostly studied by specialists in Latin America, the study of populism was suddenly very much in vogue.

I was in Spain on a fact-finding mission. The Guardian’s editor-in-chief, Katharine Viner, and her deputy, Paul Johnson, had given me the green light to run a series exploring the rise of populism. The meeting in Segovia was an opportunity to consult experts on the subject that I had been grappling with for some time.

In my previous Guardian roles – as Washington correspondent and San Francisco bureau chief – I had reported both on Trump’s extraordinary ascent to the White House and the similarly stunning success of Bernie Sanders, a once-fringe independent senator from Vermont who transformed Democratic politics.

I had also spent months trying to understand how dominant technology platforms such as Facebook, Twitter and YouTube – and the wider “attention economy” that makes them hugely profitable – were rewiring politics and creating an uneven playing field that might be tilted toward populist candidates.

Back in Europe, I recognised echoes of what I’d seen in America – populists on the left and right (although more so on the right) were gaining traction in countries as varied as Italy, Turkey, Sweden, Austria, Spain and Poland. Five of the world’s largest democracies were run by them, including India, Brazil, Mexico and the Philippines.

I learned two things from my trip to Spain. The first was that scholars who specialise in populism are exasperated at how frequently the term is overused or misapplied. The problem is especially acute in the UK, where commentators often use the term to refer to far-right, nativist politics, or use it as a shorthand for charismatic demagoguery.

My second discovery was that while lots of useful research has been conducted on populism, it was limited – and there was surprisingly little in the way of empirical research tracking populist discourse, campaigns or parties.

In the days that followed I worked with Mark Rice-Oxley, the Guardian’s special projects editor, to sketch out the kinds of questions a series on populism would ask. What is populism, exactly, and where in the world is it taking root? Which parties and political leaders deserve the populist label, and what happens when they get into power? Perhaps most difficult of all: is rightwing populism, in particular, really on the rise, and, if so, why might that be?

Facebook Twitter Pinterest The New Populism series established that the number of populist world leaders has more than doubled since the early 2000s. Composite: The Guardian Design Team

None of these questions have a simple or uncontested answer. This was always going to be an exploratory project helping us (and you, our readers) better understand populism, even if that meant there were no definitive conclusions. But we thought it was important to stay faithful to a broadly accepted definition of the p-word. Simply put, populism is a language that frames politics as a battle between the will of ordinary people and corrupt or self-serving elites, and can exist on the left or right.

A few weeks later, we launched The New Populism, a series of essays and reported dispatches from locations as varied as Rio de Janeiro, Istanbul, New Hampshire, Venice, Johannesburg, Delhi, Zurich, Glasgow and Hartlepool.

The most novel parts of the series were the research projects we conducted in partnership with political scientists, including several I met in Segovia. We created a partnership with Team Populism, a global network of political scientists, to analyse the hundreds of speeches by presidents, prime ministers and chancellors of 40 countries across the world. A grant from theguardian.org, a US-based non-profit group, enabled us to hire 46 researchers who were trained to identify populist discourse in speeches.

Brexit is only part of the story of the decline in trust in British elites that has a long roots

The result was the Global Populism Database, the most up-to-date, comprehensive and reliable repository of populist discourse in the world – and the basis upon we which we were able to report that the number of populist leaders has more than doubled since the early 2000s.

Team Populism also helped us build a data visualisation of how reliant Trump is on a teleprompter for his particular brand of rather inconsistent populism – and a hugely popular online quiz that allowed 1.5 million Guardian readers to answer several questions to discover how populist their own views were.

In another collaboration, we coordinated a team of 35 political scientists to peer-review a list of past and current populist parties in Europe. Our academic partners later helped us design and analyse a major global survey in partnership with YouGov, polling respondents in 23 countries to test populist sentiment across the world.

Facebook Twitter Pinterest Paul Lewis in June 2019. Photograph: Alicia Canter/The Guardian

The New Populism series finished last month, culminating in European parliamentary elections in which populist parties secured almost one in three votes. Nowhere was the success of populists more stark than in the UK, where Nigel Farage’s Brexit party triumphed in last month’s election by capitalising on discontent over the failure to enact the result of the EU referendum.

Brexit is only part of the story of a decline in trust in British elites that has long roots, as the sociologist Will Davies explained in one of a host of populism-themed essays we published in the Long Reads section.

It used to be said that our first-past-the-post electoral system was a bulwark against insurgent populist parties. That may prove to be true. But the language of populism is everywhere in British politics these days. You hear it in the frequent vilification of political elites, the romanticisation of ordinary or “real” people, the attacks on the media and the idea that politics is no longer just a battle of ideas but a winner-takes-all conflict between good and evil.

It can pop up in sometimes unexpected places – from Theresa May’s speeches, to the lexicon of those arch-Remainers who insist a “people’s vote” is the only legitimate response to a Brexit stitch-up contrived by powerful cabal of crooks and liars. Our six-month focus on populism has come to an end. But it seems inevitable that populism will be a subject we continue to report on for months and years to come.