The chancellor, Philip Hammond, has been accused of being blind to the scale of poverty in the UK after he dismissed a UN report that found the government’s austerity programme had caused misery for many Britons.

The report by the UN’s rapporteur on extreme poverty and human rights, Philip Alston, said 14 million people in the UK live in poverty and 1.5 million are destitute. It claimed the government’s “punitive, mean-spirited, and often callous” austerity policies were often intended to bring about social re-engineering.

On Monday, Hammond balked at the claims, saying: “I reject the idea that there are vast numbers of people facing dire poverty in this country. I don’t accept the UN rapporteur’s report at all. I think that’s a nonsense. Look around you; that’s not what we see in this country.”

He told the BBC’s Newsnight programme: “Of course, there are people struggling with the cost of living. I understand that. But the point is that we are addressing these things through getting to the root causes.”

Responding to his claims, the shadow chancellor, John McDonnell, said Hammond was insulated from the effects of his government’s policies because of his wealth and denounced him for callousness.

“Multimillionaire Hammond lives in a different world to the rest of us. He displays a brutal complacency about the scale of poverty and human suffering his austerity programme has created,” McDonnell said. “Heartless, without compassion or any sense of humanity, after these remarks he demonstrates he is not fit to hold office and should consider his position.

“It’s not just the United Nations; Human Rights Watch and others have commented on the poverty and inequality under this government. Meanwhile, Philip Hammond’s colleagues compete with each other to promise more tax cuts, showing how utterly out of touch the Conservatives are with the problems of today’s society.”

In the report released last month, Alston said the UK had violated its human rights obligations through sustained and widespread cuts to social support. He said the scale of poverty in the UK was “patently unjust and contrary to British values” and labelled it a “political choice”, rather than an economic necessity.

On Monday, a source close to Hammond said: “The chancellor was referring to Philip Alston’s report on poverty, which others in government have described as ‘barely believable’ documentation and based on a short period of time in this country. It doesn’t reflect where we are now, with the number of people in absolute poverty falling … since 2010, nor does it paint an accurate picture of our approach to poverty.”