A government taskforce announced in June to help make safe privately owned tower blocks wrapped in combustible Grenfell-style cladding has still not started work, the housing secretary, James Brokenshire, has admitted.

Only five of the 183 high-rise private apartment blocks found to have unsafe cladding have so far been repaired as disputes continue between freeholders and leaseholders over who should pay.

Figures published by the government on Wednesday showed that plans remain unclear on 50 of the buildings identified, even though they are considered unsafe. Not one of the 27 hotel towers identified with Grenfell-style aluminium composite material (ACM) cladding has yet been fully remediated.

In June, two months after he took over the response to the Grenfell disaster that claimed 72 lives, Brokenshire told parliament that, in light of the pace of repairs in the private sector, a joint expert inspection team was being convened to help councils handle the problem.

“This team will support local authorities in ensuring and where necessary enforcing remediation of private sector high-rise residential buildings with unsafe ACM cladding systems,” he said.

However, almost six months on, he was asked in parliament this week by the shadow housing secretary, John Healey, who the members of the taskforce were. He declined to answer, but said instead: “The joint expert inspection team is being established to support the next phase of work.” Officials admitted it would not start work until next year.

Labour has accused the government of dragging its feet over the removal of cladding on thousands of homes and its admission that the taskforce has not been set up as promised came just days after Brokenshire indicated he had lost patience with the speed of the private sector’s own response. He gave councils the power to strip the materials off themselves and reclaim the multimillion-pound cost from the landlord.

Healey said it was shameful that hundreds of buildings with Grenfell-style flammable cladding still had not had it replaced.

“Theresa May promised to do ‘whatever it takes’ to keep people safe, but her government has dragged its feet at every stage,” he said. “Conservative ministers must now set a firm deadline by which all buildings will have dangerous cladding removed and replaced – and take the urgent action needed to make it happen.”

A spokesperson for the communities secretary said: “Since the team was announced, private sector buildings where there is no plan or intent to replace cladding have fallen from over 200 to less than 70. The joint inspection team will begin work in the new year focusing on taking action in the remaining cases where owners are refusing to make their buildings permanently safe.”