Some years ago, I talked to Andrew Dilnot, then principal of an Oxford college, now head of the UK Statistics Authority. He picked up a copy of the Guardian front page, jabbed his finger at the figure $25bn, which was highlighted in a panel, and asked: "Is that a big number?" I looked at him blankly. He said newspapers, particularly upmarket ones, were full of numbers but, in many instances, neither journalists nor readers could explain their significance. "Numbers are just a particular class of words. There isn't any other class of words in a paper that we wouldn't ask ourselves what they mean."

We often hear politicians quoting numbers, but what do they mean? In March, a Conservative party press release, faithfully reported in the Sunday Telegraph, claimed "nearly a million people" had come off incapacity benefit rather than face new medical tests for what is now called the employment and support allowance. The press release intended us to think 1m was a big number – "more than a third of the total", it stated – though the true figure was 878,300. To explain the meaning, it quoted the party chairman, Grant Shapps. The figure was a vindication of the government's stricter policies on benefit claimants, he said, and a demonstration of "how the welfare system was broken under Labour". It showed, we were supposed to deduce, the scale of malingering before the coalition put a stop to it.

But the big number was – there is no other word for it – a lie. Dilnot, now responsible for protecting the integrity of official statistics, exposed it as a lie this week, albeit using mild Whitehall language in letters to Shapps and Iain Duncan Smith, the work and pensions secretary. The 878,300 alleged malingerers had never received incapacity benefit. They were new claimants, aggregated over three-and-a-half years. Many (probably most) withdrew their claim because they recovered from their condition or found a new job. In 2011-12, out of 603,600 established benefit claimants referred for the new medical tests, just 19,700 (3.3%) withdrew before taking them. That figure – which most of us would think small – represented the true scale of people pretending to be sick.

This is not the first time Dilnot has issued reprimands for misuse of numbers or, to put it more bluntly than he would, the quotation of bogus figures. The prime minister himself was rebuked in January for stating that the coalition was "paying down Britain's debts" when the national debt had risen from £811bn to £1.1 trillion. Jeremy Hunt, the health secretary, was told in December to withdraw his claim that NHS spending had risen in real terms "in each of the last two years". Last month Duncan Smith was on the naughty step for claiming that, as a result of the new benefits cap, 8,000 people had moved into jobs. This was "unsupported" by official statistics, Dilnot ruled. Last year Michael Gove, the education secretary, was criticised for claims that, under Labour, tests had shown British children falling steeply in international league tables. UK samples for tests in 2000 and 2003 were inadequate, Dilnot wrote; it was not therefore possible to make "trend comparisons" with later tests in 2006 and 2009.

Politicians – like journalists, campaigners and even academics – habitually quote figures selectively, seizing on those that support their case, ignoring those that don't. That is human nature. We cannot expect ministers to examine all available evidence dispassionately every time they speak or write. No doubt they also make genuine errors, misunderstanding, misreading or failing to check statistics.

But the examples above are surely deliberate attempts to mislead the public. It is not a matter of accurate figures being taken out of context, but of making false statements about what official statistics show. (Labour may have been equally guilty of such behaviour, but it was rarely properly highlighted because the UK Statistics Authority was not established until 2008 and Dilnot did not take charge until last year.) Unfortunately, there is no price to pay. The "nearly a million" figure will stick in the public mind. Dilnot's demolition of the Shapps claim was not widely or prominently reported.

The quotation of statistics is fundamental to modern political debate. Parties compete, not so much on ideology or even policy, as on their competence to manage the nation's affairs. Most voters would struggle to distinguish between a Labour NHS reform and a Tory NHS reform, a Labour academy and a Tory academy, a Labour "crackdown" on benefits and a Tory "crackdown". They look for evidence that things are going well and politicians respond by quoting hospital waiting times, GCSE success rates, numbers coming off benefit, and so on. We know politicians cherry-pick the figures, wrench them out of context, round them up or down, but we should at least have confidence that they aren't making them up.

Perhaps, as is often suggested, better maths and statistics teaching in schools would help us make more sense of the figures. But we cannot all be expected to scrutinise the raw official statistics to verify everything we are told, not least because the Office for National Statistics website is virtually unnavigable. Without some faith in ministers' veracity, public trust in democracy withers still further.

Can anything be done? The public administration select committee has proposed that Dilnot take greater control over the collation and publication of departmental statistics, and over how they are publicised. It has also suggested that ministers should not have automatic access to official figures before they are released, because it allows them to put out their own "spin" in advance. These changes would be an improvement, but ministers will continue to offend until they have reason to fear the consequences of making false statements.

Nearly all ministerial resignations are connected with not telling the truth: submitting false expenses, covering up a speeding points swap, receiving favours from lobbyists. But telling untruths about official figures is somehow regarded less seriously. Dilnot should have the power, in the worst examples, to require a full Commons censure debate on a minister's conduct – with an expectation that, if he or she failed to offer an adequate defence or show contrition, resignation would follow. That would guarantee press attention and ministerial trembling. Big lies about big numbers require big deterrents.