Image copyright Thinkstock

The issue of tax avoidance is currently massively controversial. But what drives affluent individuals who could reduce their bills from the taxman, but choose not to?

Everyone, Conservative peer Lord Fink told an interviewer, "does tax avoidance at some level".

Labour leader Ed Miliband, who criticised Lord Fink, himself faced accusations that he had avoided inheritance tax on his parents' home. Labour have subsequently said "no tax was avoided".

There's an assumption among many that the wealthiest people often avoid paying the top rate of tax on all of their income. That rate is 45% for all income over £150,000 a year.

But there are public figures who say they do pay the full amount of tax they owe.

These include comedian Ricky Gervais, who described tax exiles as "unsavoury", presenter Graham Norton, who has said "I should pay my tax because I can afford it," and author JK Rowling, who said it would have been "contemptible" for her to leave the country for lower tax rates after finding success.

Dragon's Den businesswoman Hilary Devey once described tax avoidance as "disgraceful". Other well-off Britons who have spoken in similar terms include Lily Allen, Robson Green, Frank Skinner, Ricky Tomlinson and Superdry co-founder Julian Dunkerton.

Image copyright Getty Images

It's a stance that attracts praise, but also bafflement and scepticism. There's an assumption that people who have become rich will do everything they can to stay rich, minimising their outgoings as assiduously as they maximise their earnings.

But it's fairly simple, says billionaire John Caudwell, the former head of the retailer Phones 4u - paying tax is "a moral issue". Caudwell speaks out forcefully against tax avoidance and, according to the Sunday Times Rich List, handed over almost £250m to the Exchequer in the four years to 2014.

Image copyright Getty Images

He says his involvement with philanthropic projects - he founded the charity Caudwell Children - brought home to him the consequences of avoiding tax. "The people that are being robbed are the poor, the criminal justice system, the health system, the welfare state, all the things that combine to make Britain what it is."

There are more prosaic reasons, too. Many tax avoidance schemes - as distinct from tax evasion, which is illegal - involve living outside the country for much of the year, something that Caudwell, who has homes in Staffordshire and London, says has never appealed to him.

Plus there is the sheer byzantine complexity of avoiding tax.

Mark Constantine, co-founder of the cosmetics chain Lush, says he, like Caudwell, pays the 45% rate. Constantine recalls spending three days trying to understand one avoidance mechanism that was suggested to him, before being nagged by "the uneasy feeling that it's illegal" and the fear that HMRC would come knocking if he signed up.

He says he'd rather devote all that time and effort to his business. "It involves an awful lot of work. There's a moral element to it, but also I just can't see the point."

Both businessmen agree that their attitudes towards taxation put them in a minority among the UK's wealthiest people.

Civic-mindedness may not always be the only concern at play. "Some people are worried about the reputational risk," says John Christensen of the Tax Justice Network. "They don't want to lose sleep at night because a tax avoidance scheme might unravel and put them on the front pages", even if this is not their sole or even principal motive.

It's true that, with austerity measures being felt, it's not a good time to be known as a tax avoider.

Celebrities like Jimmy Carr, Gary Barlow and Chris Moyles have been widely castigated over their tax arrangements, while multinational corporations deemed to be dodging HMRC's grasp are routinely the focus of protests and threatened with boycotts.

The cost of tax avoidance

Image copyright Getty Images

Tax avoidance schemes cost the UK £3.1bn in 2012-13 (the most recent figures), the HMRC says

The HMRC says: "The avoidance tax gap is an annual estimate of the tax lost due to avoidance schemes that are subject to HMRC scrutiny and frustrate the intention of Parliament."

£1.4bn of that was through income tax, national insurance contributions and capital gains tax

£1.3bn was in unpaid corporation tax

£0.1bn was lost through VAT and the remaining £0.2bn was through other direct taxes

Source: HMRC

According to a YouGov survey published in November, 63% of British people said they believed the moral duty to contribute to public services was stronger than the right to keep the money one earns. In the US, only 37% agreed.

Attitudes may have hardened since then, following revelations that banking giant HSBC helped wealthy clients across the world evade hundreds of millions of pounds in tax.

It's true that different people who use the term "tax avoidance" mean very different things by it, as Shadow Chancellor Ed Balls found when he was drawn into a row about whether one should get a receipt for hedge-trimming services and was subject to a newspaper investigation into how he pays his window cleaner.

Income tax - who pays what?

Basic rate 20% - £0 to £31,865

Higher rate 40% - £31,866 to £150,000

Additional rate 45% - over £150,000

Most people are entitled to earn the first £10,000 tax free

Source: HMRC

Any public figure who takes a stand against tax avoidance, indeed, risks having the minutiae of their financial affairs scrutinised for evidence of hypocrisy.

Judith Freedman, professor of taxation law and fellow of Worcester College, Oxford, says this reflects the fact that the phrase "tax avoidance" is defined too loosely.

"Everyone does what they can to reduce the tax payable within the law," she says. "I don't think you should castigate the people who have responded to the incentives."

For others, there is no ambiguity. Caudwell says it's "complete nonsense" to compare ordinary people paying into Isa savings accounts and pensions, which are subject to tax breaks purposefully created by parliament, with the exploiting loopholes sought out by accountants.

Image copyright Getty Images

But as Freedman points out, the more elaborate arrangements often turn out not, in fact, to be legal.

The risk of negative publicity also looms large. Gary Barlow and a slew of other stars were named as investors in a scheme called Liberty, which HMRC is challenging.

Others might, in future, conclude any potential savings from such an initiative would be outweighed by the headlines they would attract if it fell foul of the authorities. Then take the path of least resistance and cough up.

"It's weird that we're even talking about it," says Constantine. "Paying your taxes is what you're supposed to do."

More from the Magazine: Cash-in-hand

Image copyright Thinkstock

Treasury Minister David Gauke said in 2012 that paying tradesmen cash in hand to avoid tax is "morally wrong"

The black economy is thought to cost the exchequer some £2bn a year

The minister said there was nothing wrong with paying in cash, but doing so with the purpose of avoiding tax was wrong

How 'morally wrong' is it to pay cash-in-hand?

Subscribe to the BBC News Magazine's email newsletter to get articles sent to your inbox.