George Osborne is making political capital out of seeking to save £4bn on the benefits bill, happy for those making the claims he's targeting to be called lifestyle choice fraudsters and layabouts – all, supposedly, because of the need to tackle the hole in the government's deficit. But he wouldn't need to make these cuts if he tackled the biggest category of fraud in the UK economy – that of tax evasion.

The tax gap is real. It's the difference between the tax that should be collected from the UK economy if HM Revenue & Customs knew everything that was going on and the tax it actually collects. HMRC claims the gap is £40bn a year with well over £30bn of that being tax evasion and a much smaller part – less than £5bn – being tax avoidance. The difference between the two is important. Evasion is illegal – it's fraud, in other words. Avoidance is the smart trickery my colleagues in the accountancy profession play.

The trouble is HMRC has these estimates wrong. I estimate that tax evasion – the issue I'm concerned about here – costs about £70bn a year. My estimate is based on the rate of VAT evasion that HMRC admits to – which I calculate to be an average of about 13.7% over the past seven years.

That means more than £1 in every £8 of VAT due in the UK is evaded. Shockingly, the World Bank has recently confirmed in a study of the size of the cash economy in 162 economies that they agree almost exactly with this ratio for the UK, suggesting on average that the UK shadow economy is about 13.5% of GDP, and on an upward trend.

Despite this evidence HMRC refuses to recognise that if VAT is evaded at the rate its admits then it follows that this proportion of income tax, corporation tax and national insurance is also evaded – which is an untenable position on its part. No business person puts cash in their pocket to evade VAT and then declares income tax on the wages and profits paid out of that cash. Those other taxes are evaded as well, and by as much – if not more – than VAT, simply because VAT doesn't apply to all businesses but income tax and national insurance always do. And as a result £70bn is lost to tax evasion a year. That's enough to pay our way out of our current financial crisis.

But this does not happen by chance. This cash has to get into the hands of fraudulent traders – and not much of it comes from them trading with each other. Most of it comes from the public who, when offered a deal for cash take it. Builders are the classic case everyone points too. But so too are after-school tutors these days. And nannies and domestic cleaners paid cash in hand. And those who trade through car boot sales. And even people who trade on eBay and "forget" to tell HMRC. The list of ways cash creeps out of the tax system and into the shadow economy are numerous.

And the fact is that cash on this scale does not just come from those committing benefit fraud. Cash on this scale comes from the middle and upper classes – Guardian readers among them, no doubt. Every time you pay cash, in these ways and more, you contribute to the tax gap. You deny the government the cash it needs to preserve public services. You facilitate fraud, even if you're not guilty of it. You undermine the NHS. And your children's education, and all those other services you value. And you help deny benefits to those who need them. The joy of tax is that it pays for all these things. Tax evasion denies them to us.

If there is a "big society" – not in the way Cameron describes it but in the way we believe in the society we live in and enjoy the services our state provides – then the cash economy directly undermines it. That's the real consequence of the cash deal to save a bit on the cost of cramming for an A* GCSE.

And that's a challenge for all who do believe in society, the rule of law, the value of government services and the democracy we enjoy. Are you willing to pay by cheque or card, to demand a receipt, to operate PAYE on your domestic staff, to clamp down on tax evasion, and say so? It's a choice you can make. You can choose to pay tax. Will you do that to keep the services you want?

Ask yourself that the next time you could evade tax. And live with your conscience if you contribute to the tax gap – a gap we, and those who rely on the state, can no longer afford.