Oversized neon Olympic rings illuminate Vancouver harbour while a giant mural of a snowboarder mid-trick welcomes visitors to the airport. In the city centre, a sleek steel-and-glass Olympic clock is counting down to next month's winter games.

But for 51-year-old Wayne, a homeless drug addict, looking up at the snowcapped mountains where the downhill competition runs will be fills him with dread.

"We're all going to be cleared out of here before the Olympics," he said, wrapped in a flimsy sleeping bag and clutching a bag of bottles plucked from street bins which he will exchange for money. "The clean-up will happen – they all want to hide the city's black eye, right?"

That black eye is the Downtown Eastside (DTES), one of the most highly visible and divisive parts of the Canadian city's involvement with the Olympics. The area is both ghetto and historic community. It boasts a high concentration of single-room accommodation and cardboard-and-shopping-trolley "homes" for Wayne and many of the region's other 2,660 homeless people.

With Canada's poorest postcode sitting within a city regularly ranked as the most livable in the world, the city's leaders know Vancouver is preparing for a barrage of scrutiny: yards down the street, the British Columbia provincial government has even set up an information centre, DTES Connect, to try to showcase its work in the area to the world's media.

Ever since the games were awarded to Vancouver, population 611,000, and the nearby resort of Whistler in 2003, resistance has focused on housing. Ceremonies have been disrupted, and officials of the Vancouver games organising committee (Vanoc) have been threatened with symbolical "evictions" from their homes by Anti-Poverty Committee protesters. Before Christmas, more than 100 campaigners blocked rush-hour traffic, shouting and waving banners saying "Don't kidnap the homeless for the Olympics".

The anxiety stems from a recent provincial government law empowering the police to force rough sleepers into shelters in extreme weather, a move which homeless groups appear to view as an Orwellian effort at civic image control. Police officers have been told to use only "non-forceful touching" in implementing the Assistance to Shelter Act, but that has not stopped critics calling it the Olympic Kidnapping Act.

"Word on the street holds that this is a way to clean-up during the Olympics," said Tracey Axelsson, project manager for Home for the Games, a non-profit organisation for residents to host visitors for the Olympics and split the rental income with homeless groups.

David Eby, the executive director of the BC Civil Liberties Association, said: "I don't feel that there is any question among most people who have been following the homeless issue in Vancouver that this act is targeted at giving police a tool to remove homeless people from high-visibility tourist areas."

Although proposed bylaws against sitting or lying on the pavement were rejected, he thought there would be further attempts to displace the homeless.

For three decades, the port has been a magnet for sex workers driven out of wealthier neighbourhoods, addicts and people released from mental wards shut in the 1990s. Methamphetamine and crack-cocaine users shoot up on the street yards from a police station. A relatively moderate climate and the concentration of needle exchange, safe injection sites and sex worker drop-in centres are further draws.

Where Vancouverites are most divided is whether there is any connection between spending more than £3.5bn on the Olympics and the city's homelessness problem: according to the most recent official homeless count, numbers soared 137% between 2002 and 2008.

"There are those who argue," Eby said, "that we shouldn't be spending billions on a two-week party until we've dealt with the fact that people are freezing and starving in our streets."

Chris Shaw is a professor of ophthalmology at the University of British Columbia, and a spokesman for 2010 Games Watch, part of the Olympic Resistance Network. He thinks the benefits of the games are nothing more than empty promises. "There are just greater attempts to 'solve' the problem with more policing and expelling the homeless," he says.

All the main partners involved in the games deny this, pointing instead to the planned affordable housing in the athletes' village, residents being hired to collect rubbish for recycling during the games, and multi- million-dollar revitalisation plans for the area.

"We are not going to try to 'cleanse' our streets," Kerry Jang, a local councillor, said. "We're investing in the community, and we feel very strongly that things are getting better. But it's not going to happen overnight."

Some homeless groups have seen the games as a catalyst for action.

"I can't say for sure if the same level of government support towards reducing homelessness would have occurred without the Olympics," said Michelle Clausius of Covenant House, a group that supports homeless youngsters, "but I know that the issue is much more in the spotlight than it was prior to the awarding of the games."

As for Wayne, after six years of sleeping rough he has his own opinion on the long-term solution: "I despise the Olympics, but you know what the answer is? Just pull down the whole of the Downtown Eastside, with its bedbugs and cockroaches. Just destroy it all."