First, they started building a four-bedroom house without a permit.

But what two brothers ended up building wasn’t the single-family home they’d promised — it was a 21-room student residence, also without a permit.

Now the building sits empty because the City of Toronto refuses to let anyone occupy it. And last week a court ruled against the brothers, who wanted a permit for the now-completed building.

For five years, Steve and Luke Williamson have been embroiled in a dispute with the city over the construction on Victoria Park Ave. near Finch Ave.

“It’s a witch hunt,” said Steve Williamson, who admits he made a mistake but doesn’t feel the city is helping reach a resolution.

“They built without a permit,” said Tim Carre, a lawyer for the city. “Then once they got a permit they did not build within accordance of that permit. They’ve had people in the building occupying it in a manner that was contrary to the building code, contrary to the fire code.

“They’ve had every opportunity to bring it into compliance and they’ve chosen thus far not to.”

Williamson said he has become disillusioned by the slow pace of the permit and inspection process.

“I don’t feel that they’re trying to get anything constructive done,” he said. “They’re just trying to stick it to me every which way possible.”

On Oct. 3, an Ontario Superior Court judge dismissed an appeal brought by the Williamsons after the city denied their attempt to get a retroactive permit for the already completed 21-room building.

In his decision, Judge Sean Dunphy laid out a timeline of events that followed the Williamsons through five years of broken bylaws and shirked zoning rules:

In June 2011, the Williamsons applied for a permit to build a four-bedroom home on Victoria Park Ave. just south of Finch Ave. E., on a property owned by Luke Williamson.

Approval for the permit was withheld because of some minor problems with the planned height and length of the house.

The Williamsons began construction anyway, ignoring two municipal orders in October 2011 to stop work.

In June 2012, an Ontario Superior Court judge ordered the building to be demolished, but then stayed his order to give the Williamsons another chance to get their original building permit approved.

In June 2013, after a failed application and a successful appeal, Williamsons were granted their permit for a two-storey, single-family residence with four bedrooms.

But when city officials tried to inspect the property a short time later, they were denied access.

Eventually the city obtained a search warrant and, in June 2014, police and municipal officials found a “substantially complete and mostly furnished hotel,” which was “substantially different from the interior layout of the building as represented in the drawings that formed the basis of the . . . building permit.”

Subsequent inspections found that the property was already in use as a rental property.

In October 2015 a judge decided that the building could not be occupied until more inspections were conducted and changes were made to make the building into what its permit called for.

Loading... Loading... Loading... Loading... Loading... Loading...

In November 2015, the Williamsons applied for a permit for the building that now exists — a 21-room building.

That application was rejected in June 2016 and the Williamson appealed, leading to the Oct. 3 denial of that appeal.

Steve Williamson told the Star that when he and his brother filed their first permit application in 2011, they intended to build a four-bedroom family home, as promised.

But then, Williamson said, he was approached by local high schools who said they had international students in need of housing and wondered if the brothers were building rental properties.

“So I was like, ‘You know what? I could make some more bedrooms,’ ” said Williamson.

“From a business standpoint . . . if there’s a need for it, then it’s like, ‘OK, then go this way.’ ”

The city said students from the nearby William Academy private school were found living in the Williamson’s building in 2015. A spokesman for the school said it was the Williamsons who approached them about renting rooms to students.

Williamson said he still believes there is a need for student rentals in the community and that most such residences fail to comply with the building code.

“Show me a legal student rental. Show me one,” he said. “If any place should be legit, I think it should be one that’s not hard on the eyes, you’re on a major street, close to the bus stop.”

Carre said the city has gotten a court order to stop the Williamsons’ property from being occupied until it is brought into compliance with building code.

That means the Williamsons can either alter the building to fit their four-bedroom family residence permit, which is still valid, or they can try to have the property zoning changed.

Williamson said he is not sure which avenue he will take, but that either one would be prohibitively expensive.