A fire has severely damaged the Traveler's Inn in Victoria, displacing dozens of long-term tenants.

Victoria fire crews were called out just before midnight. Residents were evacuated from the three-story building and the exterior of the structure was damaged.

Acting battalion chief Wayn Moody said fire crews thought the fire was extinguished at that time, but then around 7 a.m. the fire alarms in the building sounded again.

"When I came out into the hallway, it was full of smoke," said Claude Bishop, 80, who has lived at the hotel for two years. "I got out just in time."

Acting Batallion Chief Wayn Moody speaks for the Victoria Fire Department. (Megan Thomas/CBC)

The Victoria Fire Department called in crews from the neighbouring municipality of Saanich to help knock down the blaze.

Moody said all of the dozens of tenants made it out of the building, but a couple of people were taken to hospital to be treated for smoke inhalation.

As many of 60 people have been displaced from the building and are receiving emergency support from the City of Victoria.

"We are having challenges as well from some of the people," Moody said. "They have medical issues. We are sending runners in to get their medicine and stuff like that."

Moody said he believes debris in a truck parked nearby went up in flames, which then spread up an old chimney chase in the building, but the cause remains under investigation.