There are a lot of idiots around — just look at the front benches of the House of Commons. But why do the idiots reach such elevated positions?

Many onlookers feel sure that the successors to David Davis, the UK’s former Brexit secretary, and Boris Johnson, ex-foreign secretary, cannot possibly be worse than the men they replace. Mr Davis was at least dignified in his resignation. Mr Johnson is widely regarded as the most disgraceful occupant of the post imaginable. And yet both men departed at the hour of their choosing: they were incompetent yet unsackable.

Part of this displays the awful dynamics of British politics today. But it also reflects something more universal. In 1969, Laurence Peter, a professor of education, and Raymond Hull, a playwright, published a management book that became a bestseller and an enduring classic: The Peter Principle. (UK) (US)

The Peter principle states that “every employee tends to rise to his level of incompetence”. If someone is good at her job, she’ll be promoted into a job that demands different skills. If she’s good at the new job too, she’ll be promoted again, requiring yet another set of skills. One day, she will arrive at a job for which she is wholly unsuited, and there she will stick. Since when did a manager ever get sacked for anything?

The Peter Principle is satire: it mocks management and it mocks books about management. It is striking, then, that most people take it quite seriously. The Harvard Business Review has published numerous straight-faced responses.

Two questions, then: is the Peter principle true? If so, what can we do about it?

We should acknowledge that the Peter principle may be an illusion. The top jobs are difficult to do well; they can make monkeys out of capable people. We shouldn’t be surprised if it is easy to call to mind examples of business or political leaders who have struggled. That would not by itself mean there are systematic errors in the way people rise to the top.

People were not promoted for behaviour that might seem correlated with managerial ability — in particular, those who collaborated with others were not rewarded for doing so

But this year, the economists Alan Benson, Danielle Li and Kelly Shue published what may be the first detailed empirical investigation of the Peter principle. Profs Benson, Li and Shue used data from a company that provides performance management software to sales teams; the data cover 214 firms, more than 53,000 workers and more than 1,500 promotions. This data set was ideal for identifying highly effective sales staff, and also provided enough information to evaluate what happened to a team once a hotshot salesperson was promoted into the role of team leader.

The authors of the paper discovered that the best salespeople were more likely to be promoted, and that they were then terrible managers. The better they had been in sales, the worse their teams performed once they arrived in a managerial role.

What’s more, people were not promoted for behaviour that might seem correlated with managerial ability — in particular, those who collaborated with others were not rewarded for doing so. What mattered were sales, pure and simple.

In short, Professor Peter was right. Brilliant people are promoted until they become awful managers. Perhaps employers are using the prospect of promotion as an incentive, and are willing to accept the collateral damage caused by all these terrible managers. If so, they are probably making a mistake. It would be better just to encourage the star salespeople with cash bonuses instead.

There is a more radical approach. One of my favourite IG Nobel Prizes (the IG Nobels reward “achievements that first make people laugh, then make them think”) was awarded to two physicists and a sociologist — Alessandro Pluchino, Andrea Rapisarda and Cesare Garofalo — who wondered how organisations might evade the logic of the Peter principle.

If performance at one level of a hierarchy is uncorrelated with performance at the next level up, the best strategy is simply to promote the very worst people. Nobody knows whether they will make good managers, but at least they will no longer be dreadful staff — or as Dogbert in the cartoon strip Dilbert put it back in 1995: “Leadership is nature’s way of removing morons from the productive flow.”

There are two difficulties with this approach: first, it may be too extreme to assume that no skills at all carry over from one job to the next; second, if the reward for failure is promotion, then the likely response is an organisation full of people bent on sabotage. So Profs Pluchino, Rapisarda and Garofalo suggest a compromise: promote people at random.

This may be the best response to a world where leaders stick around until they are ready to depart. But there is an obvious alternative: when people are not up to the demands of their job, we should not wait for them to resign. They should be sacked — or, perhaps better, demoted back to the roles where they once flourished. A mistake is regrettable, but stubbornly sticking to the mistake is far worse. Just ask the British electorate.

Written for and first published in the Financial Times on 20 July 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now (or very very soon) in paperback – feel free to pre-order online or through your local bookshop.