Rising rough sleeping in Britain is not the result of government policy but is being driven by factors including the spread of psychoactive drugs such as spice, growth in non-UK nationals on the streets and family breakdown, the housing secretary, James Brokenshire, has claimed.

The number of people sleeping rough has more than doubled since 2010 to 4,751 according to the government’s own figures. The homelessness charity Crisis believes that this is a fivefold underestimate and that 24,000 people will be sleep on the streets, in cars and in tents. Sofa-surfers make up a further 68,000, according to Crisis.

But Brokenshire insisted the growing problem is not a political failure, even though charities which run hostels and advice lines believe that caps on housing benefit and welfare sanctions introduced as part of austerity policies have been key factors driving rises in homelessness every year since the Conservatives took office in 2010.

“I don’t see it in those terms,” Brokenshire said. “I see it as a combination of concerning elements in terms of addiction, family breakdown issues. The thing that struck me over recent months in speaking to some of the LGBT charities in terms of young people, because of their sexuality, being thrown out of home.”

Highly visible increases in tents in shop doorways in towns and cities and hidden rural homelessness mean this year’s government figures, taken during a census last month, are likely to show an eighth consecutive annual rise in homelessness in England.

Brokenshire said: “No one should ever have to face a night on the streets” as the government announced £4.8m on Tuesday for 11 new rough sleeper shelters in places including Derby, Liverpool, Brighton and Lincoln. But the new shelters will not open until next spring and the Met Office is forecasting that temperatures will fall across the country for the Christmas and new year period.

The prisons minister, Rory Stewart, also conceded that “far too many” inmates released after short sentences are struggling to find housing, with over a third of rough sleepers in London known to have been in jail.

The shadow housing secretary, John Healey, said Brokenshire had “his head in the sand”. Labour announced it would spend £100m to help rough sleepers in freezing weather. A count by the Bureau of Investigative Journalism found an estimated 484 people had died while homeless since October last year.

“Rough sleeping fell under Labour but has risen every year since 2010,” said Healey. “If Conservative MPs seriously won’t admit that their decisions have caused the rise in rough sleeping then they can’t fix it either. It beggars belief that ministers can’t understand that homelessness goes up if you slash funding for low-cost homes, cut back housing benefit and hostel funding, and fail to give private renters any security in their home.”

However, council house building fell to a slower pace under the last Labour government than at any time since.

Brokenshire said that “rough sleeping does not represent the values and the country that I strongly believe in” and that “from my political outlook it’s important that we are supporting the most vulnerable”.

But he played down the impact of the transition to the new universal credit welfare system amid growing evidence that delays in payments are forcing some residents into rent arrears and into homelessness.

The new system was “about ensuring there aren’t those gaps,” he said. “It is about supporting people into work.”

Crisis responded by insisting that government policies are driving up homelessness. While the charity applauded the government’s rough sleeping strategy and practical commitments to help people off the streets, it said decisions on housing benefit which make housing unaffordable for those in greatest need were making people homeless.

“Without commitments to tackle the root causes of the issue, it will continue to treat the symptoms of homelessness without finding a cure,” said Matt Downie, its director of policy.

Brokenshire cited the difficulty of helping people battling both drug abuse and alcohol addiction. He said he was working with the Home Office to tackle the rise of non-UK nationals sleeping on the streets of the capital. He is also considering ways to increase the security and stability of tenancies in the private rented sector. The loss of a private tenancy through eviction or simply being told to leave is now the leading cause of homelessness, according to Shelter, the housing charity.

Greg Beales, the charity’s campaign director, said the government must tackle that and “reverse the decline in social house-building that is leaving so many in temporary and emergency accommodation or on the street”.

Brokenshire said the Ministry of Housing, Communities and Local Government was modelling the impact of various policies on homelessness and “how we understand and look at that evidence base more clearly”.

“It is wrong to say it is this or that or the other,” Brokenshire said. “The causes of homelessness and rough sleeping are multi-layered and complex and therefore we do need to look at this in that way and ensure that councils and other agencies are getting ahead of this and preventing people becoming homeless in the first place. That is the agenda I want to move to in the new year.”

• This article was amended on 19 December 2018 to clarify the numbers of people sleeping rough and the additional number of people sofa-surfing, according to Crisis.