The housing secretary, James Brokenshire, has admitted Conservative policies may be to blame for rising levels of homelessness, appearing to row back on a statement he made earlier this month.

In an interview with the Guardian, Brokenshire had previously claimed rising rough sleeping in Britain was not the result of government policy, but was instead being driven by factors including the spread of psychoactive substances such as spice, growth in the number of non-UK nationals on the streets and family breakdown.

But on Monday, speaking to the website Politico, the minister acknowledged that Tory policies may have played a role in the huge increase in rough sleeping since 2010.

Brokenshire said the Conservatives “need to ask ourselves some very hard questions” about why so many more people are now living on the streets than when the party came to power, and admitted “changes to policy” were needed.

The comments follow the death of Gyula Remes, who collapsed in a stairwell outside parliament. The Labour MP David Lammy tweeted: “There is something rotten in Westminster when MPs walk past dying homeless people on their way to work.”

A fundraising appeal set up by staff on the parliamentary estate to support the charity StreetLink in memory of Remes has already raised more than £10,000.

The number of people sleeping rough has more than doubled since 2010 to 4,751, according to government figures.

Brokenshire did not deny cuts to welfare may have played a part in the growing problem of homelessness, and said work was under way with the work and pensions secretary, Amber Rudd, to assess where problems were. Brokenshire also revealed he personally does not give money to homeless people, although he said he buys the Big Issue when he can.

The interview appeared to contradict his remarks earlier this month, when the minister said rising homelessness was not a political failure.

“I don’t see it in those terms,” Brokenshire said. “I see it as a combination of concerning elements in terms of addiction, family breakdown issues. The thing that struck me over recent months in speaking to some of the LGBT charities in terms of young people, because of their sexuality, being thrown out of home.”

John Healey, the shadow housing secretary, said Brokenshire had “his head in the sand”. Labour announced it would spend £100m to help rough sleepers in freezing weather.