In the same week that the investigative reporter Jeremy Scahill spoke of the need for the US to "take a humility pill", we've been subjected to precisely the opposite – yet another instalment of Foreign Policy magazine's annual Failed States Index, complete with accompanying "postcards from hell" purporting to show what it's like "living on the edge in the world's worst places".

Quibbling with the many bizarre claims of the index is tempting (Kenya is "less stable" than Syria, we learn), but in the end such gripes only give credibility to this tedious yearly exercise in faux-empirical cultural bigotry. For anyone interested in actually finding out about places such as Yemen or Uganda, the index is probably the last place you'd want to go. But what's more interesting, and more helpful in understanding what the index really does, is to grasp that the very concept of the "failed state" comes with its own story.

The organisation that produces the index, the Fund for Peace, is the kind of outfit John le Carré thinks we should all be having nightmares about. Its director, JJ Messner (who puts together the list), is a former lobbyist for the private military industry. None of the raw data behind the index is made public. So why on earth would an organisation like this want to keep the idea of the failed state prominent in public discourse?

The main reason is that the concept of the failed state has never existed outside a programme for western intervention. It has always been a way of constructing a rationale for imposing US interests on less powerful nations.

Luckily, we can pinpoint exactly where it all began – right down to the words on the page. The failed state was invented in late 1992 by Gerald Helman and Steven Ratner, two US state department employees, in an article in – you guessed it – Foreign Policy, suggestively entitled Saving failed states. With the end of the cold war, they argued, "a disturbing new phenomenon is emerging: the failed nation state, utterly incapable of sustaining itself as a member of the international community". And with that, the beast was born.

What followed in the essay was a grumpy version of the history of the "third world" after 1945, in which Helman and Ratner lamented that the claims of "self-determination" made by colonised peoples had ever been established as a major principle for organising international affairs. With the fall of the Berlin Wall, Helman and Ratner argued, the time for fripperies such as state sovereignty for third world nations was over. What these failed states needed was the ever-benign "guardianship" of the western world. We westerners would keep hold of our sovereignty, of course; they would make do with something called "survivability" instead, and be grateful for it.

Helman and Ratner's piece elaborates on a well-known, but not much read, UN report by then general secretary Boutros Boutros-Ghali, which had come out a few months earlier. In his Agenda for Peace, Boutros-Ghali recommended an expanded role for the UN in resolving international crises, but insisted that state sovereignty remain an inviolable principle. This was pretty much the opposite of what Helman and Ratner wanted, but if they insisted that they were in full agreement with him, then who's to quarrel with that?

Back in the 90s, few political scientists showed any interest in the concept of failed states, and binned it on arrival. The problem was that it didn't offer any insight as a mode of analysis: a civil war is a civil war. A famine is a famine. A political crisis is a political crisis. A failed state is just rhetoric without a substantial theoretical or historical basis.

Rejected by scholars, the idea of the failed state has instead found a home within the noisy space of shallow political punditry that forms much of the national conversation. Foreign Policy offered it something of a second life by publishing its annual index from 2005 onwards, at a time when the unfolding disaster of the Iraq and Afghanistan wars, both of which had been justified as "humanitarian interventions", was painfully clear.

Unsurprisingly, given that the term was custom made to advocate for precisely such interference by the US overseas, the term also made an appearance in the literature drafted between 2001 and 2005 that created the new international norm of the responsibility to protect (R2P), a doctrine whose application by the international community so far can best be described as highly selective.

There's nothing empirical or objective about the Failed States Index, however many "stability" metrics they try to squash together. It doesn't much matter where a particular country shows up in a given year. Putting history in a league table is plainly absurd, and – when it boils down to it – the index argues the same thing every year: that the US should be a kind of global regulator to which the rest of the world must submit.

It offers a version of the world to the American public that bears no relation to reality, but works very well as a way of rationalising overseas interventions past and present.