More than 14 million people, including 4.5 million children, are living below the breadline, with more than half trapped in poverty for years, according to a new measure aimed at providing the most sophisticated analysis yet of material disadvantage in the UK.

The measure seeks to forge a fresh political consensus between left and right over how to define and track poverty, with the aim of encouraging better-targeted poverty interventions, and making it easier to hold politicians to account.

It finds poverty is especially prevalent in families with at least one disabled person, single-parent families, and households where no one works or that are dependent for income on irregular or zero-hours jobs.

What is the new UK poverty measure – and why is it needed? Read more

It was developed by the Social Metrics Commission (SMC), an independent body bringing together poverty specialists from across the political spectrum to devise a successor to the child poverty targets abolished as an official measure in 2015.

The SMC’s most significant innovation is to build core living costs such as rent and childcare into its poverty measure. This recognises that even a relatively comfortable income is no guarantee that people can meet basic material needs if it is eaten up by unavoidable weekly outgoings.

It also looks at the depth and severity of disadvantage, concluding that 12% of the total UK population is in “persistent” poverty, meaning that they have spent all or most of the last four years below the breadline. Workless families, and families that contain a disabled person, are most likely to be stuck in poverty.

There are significantly fewer over-65s in hardship compared with previous official poverty measures (11% against 16%), reflecting not just the protection of pension values in recent years, but pensioners’ lower living costs and higher levels of cash savings.

SMC’s analysis of official data finds those in hardship are more likely to have poor health and lack qualifications than those above the poverty line. But family relationships are equally strong either side of the poverty line, and the poor are significantly less likely than their wealthier counterparts to drink to excess or take illegal drugs.

Overall, the total numbers of people in poverty are marginally higher than previous figures – for example, it finds 4.5 million, or about 33%, of children are in poverty as opposed to 4.1 million (30%). But the SMC says numbers are less important than understanding the nature of poverty, and taking action to improve the lives of the poor.

“We want to put poverty at the heart of government policymaking and ensure that the decisions that are made are genuinely made with the long term interests of those in poverty in mind,” the commission’s chair, Philippa Stroud, said.

The findings are likely to result in fresh scrutiny of the impact of austerity and labour market changes over the past few years, particularly the billions of pounds of cuts and freezes to welfare and disability benefits, and the rise of insecure zero-hours work.

The commission hopes the measure will also focus politicians’ attention on tackling rising living costs – such as rent, childcare, and the extra costs of disability – rather than to solely focusing on raising incomes.

The SMC sets the poverty line at 55% of the three-year median of total household resources, as opposed to the previous threshold of 60% of median income. It says poverty is a relative concept best understood as “the extent to which people have the resources to engage adequately in a life regarded as the ‘norm’ in society”.

There was initial scepticism when the SMC was set up over two years ago – not least because Lady Stroud, formerly an adviser to the former work and pensions secretary Iain Duncan Smith, was regarded as a partisan figure because of her political links. She is chief executive of the Legatum Institute, a rightwing thinktank.

Other members include senior Conservative figure Stephan Shakespeare, who is the founder of polling firm YouGov, and another former Duncan Smith adviser, Stephen Brien, who helped designed universal credit. Alex Burghart, a former adviser to the prime minister, Theresa May, left the commission after being elected as a Tory MP in 2017.

However, the commission includes respected poverty experts from academia, and from organisations such as the Joseph Rowntree Foundation, the Institute for Fiscal Studies and the Royal Statistical Society. The former Liberal Democrat MP David Laws is also a member.

There has been broad acceptance of the SMC’s findings among poverty campaigners, who have been reassured by its rigour and independence. Both the JRF and IFS intend to use the new measure in their regular poverty analyses. The SMC is hopeful the government will adopt it as a formal measure.

The UK as a whole has been left without an official measure of poverty since the Conservative government’s controversial scrapping of Labour’s child poverty targets – although the previous measure was retained in Scotland, Wales and Northern Ireland.

Child Poverty Action Group chief executive Alison Garnham welcomed the new measure but said measurement alone would not improve life for struggling families. “What we now need is for government to move on from its denial of the problem, set targets for reducing and eradicating child poverty, and implement policies to support low-income families,” she said.

Sam Royston, director of policy and research at The Children’s Society, said: “While we would welcome these changes to how poverty is measured being included in official statistics, concrete action is needed to tackle the shameful scale of poverty among our children, with all the damage it can do to their wellbeing, education and life chances.”

The IFS has predicted the number of children living in poverty will soar to a record 5.2 million over the next five years as government welfare cuts bite, more than reversing all the progress made over the past 20 years.

A government spokesperson said: “Measuring poverty is complex, and this report offers further insight into that complexity and the additional measures that can be taken into consideration.”

The spokesperson cited reforms they said were helping people “to progress into work and then progress in work”, as well as £90bn a year spent on working age benefits and £54bn a year on supporting disabled people and those with health conditions.

They added: “Since 2010, this government has seen over 1,000 more people moving into work each and every day, the number of children living in workless households has fallen by over a third, whilst at the same time one million people have been lifted out of absolute poverty, including 300,000 children. We know there is always more to do … we will consider closely the report’s findings in full.”