In 2016 Boris Johnson and Michael Gove narrowly won the referendum on EU membership. But it turned out they had no idea how to turn their victory into a concrete policy. They had dismissed every potential difficulty as just more “Project Fear”, and it became clear they were not just doing this just because it was effective rhetoric: they had not thought through any of the major problems that implementing Brexit would create. They looked rather shocked when they won, realising that these problems they had airily dismissed would now have to be resolved.

Cameron resigned, and the Conservative Party needed to choose a new prime minister whose main preoccupation would be negotiating the terms of our exit. Their choice was Theresa May, who was known from her previous job as being non-collegiate, slow to adapt but obstinate in the views she held. These were almost the exact opposite of the qualities needed in any negotiation with a more powerful neighbour. Perhaps knowing this, she chose David Davis to handle the details of negotiation, a man who had the charm that May lacked, but who had no interest in the details, in part because he clung on to the belief that the EU would cave at the last minute.

If we cross the Atlantic, then the story is the same but more so. The hard part is thinking about an issue or decision where Donald Trump has displayed any competence. Most recently he tried and failed to appoint to the board of the central bank two people, Stephen Moore and Herman Cain, whose main qualifications were, respectively, that their predictions were always wrong, and that they ran a Pizza company.

You could perhaps put all this incompetence down to the exceptional peculiarities of Brexit and Trump. But May also appointed as Northern Irish secretary someone who didn’t realise voters there voted along sectarian lines; or Chris Grayling who, after his disastrous privatisation of the probation service, awarded a Ferry contract to a company that had no ferries, and so on. A key campaign theme of the Republican party in 2016 was to repeal Obamacare – but once Trump was elected and they had control of Congress, it turned out they had no idea what to replace it with.

Nor did this incompetence suddenly emerge out of thin air in 2016. David Cameron implemented a policy of cutting public spending in the middle of the worst recession since WWII, leading to the slowest recovery in centuries. He allowed his minister for health to implement a fundamental reorganisation of the NHS that turned out to be a disaster, at the same time as his austerity policy starved the service of funds. Of course it was also David Cameron who made a commitment to hold the EU referendum in the first place under terms that were most favourable to the Leave side.

Simon Kuper, in a brilliant article in the Financial Times, has an interesting explanation for this epidemic of incompetence. He writes how leaders like Macmillan, George HW Bush or Clement Attlee had their formative experiences in fighting WWII, while Lyndon B Johnson, Bill Clinton, and John Major had a visceral experience of poverty. They knew in their bones that government mattered. He goes on:

“But both countries have now fallen into the hands of well-off baby boomers, born between 1946 and 1964 – the luckiest members of the luckiest generation in history. These people had no formative experiences, only TV shows. They never expected anything awful or unknown to happen. They went into politics mostly for kicks.”

I’m sure Kuper is right that if our current leaders had had the strong formative experience of living with poverty or living through WWII their behaviour would have been different. In particular, they might have thought twice about using populist tropes like “the will of the people”. But surely being “the luckiest members of the luckiest generation” in history might be a necessary but not sufficient condition for being incompetent.

An interesting example here is Tony Blair*. The son of a barrister, he attended a school in Edinburgh that is sometimes described as Scotland’s Eton, and then went to Oxford University. Together with Gordon Brown he presided over an administration that championed evidence-based policy. A clear example was the decision in 2003 not to enter the Euro. The Treasury spent a year researching the pros and cons of joining the European currency, consulting widely with outside experts. The 18 background studies that effort produced are excellent examples of literature reviews or, in some cases, applied research. Although Blair was predisposed to favour entry, he was content to allow the evidence the Treasury produced to persuade him not to join.

There is of course one glaring exception to this record, and that is Iraq. The war was the idea of George Bush Jr, and it was a nonsensical response to 9/11. Most of the evidence at the time suggested that there was no connection between Iraq and al-Qaeda, and that although the war could be won keeping the subsequent peace would be very difficult. Blair followed Bush because of a simple but tragically incorrect idea – that the close UK-US alliance had to be preserved at all costs. He ignored domestic advice about the problems any post-war period would create.

The Labour government of 1997 to 2010 was not flawless by any means, but it terms of competence it is clearly better than what came later. It is hard not to see that evidence based policy protects you from many, but not all, policy mistakes.

Cameron made the commitment to a referendum in 2013 because the political imperative was to stop the rise of Ukip and possible defections from the party. The evidence was opinion polls at the time, which suggested that Leave could easily win. At a deeper level he should have realised the influence a very pro-Brexit press could have, and also that his own immigration missed targets and the rhetoric that he himself had used to justify them would beat economic forecasts in voters minds.

An ideology is a collection of ideas that can form a political imperative that overrides evidence. Indeed, most right-wing think tanks are designed to turn the ideology of neoliberalism into policy-based evidence. It was this ideology that led to austerity, the failed health reforms and the privatisation of the probation service. It also played a role in Brexit, with many of its protagonists dreaming of a UK free from regulations on workers’ rights and the environment. It is why most of the recent examples of incompetence come from the political right.

A pluralist democracy has checks and balances in part to guard against incompetence by a government or ministers. That is why Trump and the Brexiters so often attack elements of a pluralist democracy. The ultimate check on incompetence should be democracy itself: incompetent politicians are thrown out. But when a large part of the media encourage rather than expose acts of incompetence, and the non-partisan media treat knowledge as just another opinion, that safeguard against persistent incompetence is put in danger.

*It has been pointed out to me that at the age of 10, Blair's father had a stroke and lost the power of speech for over two years, meaning he could not work and his family fell on hard times. So here too Kuper's point may apply.