From the outside, the house that stands at 121 Mortimer Road in Hackney, east London, looks no different to the thousands of other decrepit old buildings scattered across the country. The roof has caved in. Three of the windows are boarded up and cracked paint peels from the wrinkled walls.

But this is no ordinary house. Since the early 1960s, the man who owns and lives inside the £1m Victorian property has been digging. No one knows how far the the network of burrows underneath 75-year-old William Lyttle's house stretch. But according to the council, which used ultrasound scanners to ascertain the extent of the problem, almost half a century of nibbling dirt with a shovel and homemade pulley has hollowed out a web of tunnels and caverns, some 8m (26ft) deep, spreading up to 20m in every direction from his house.

Their surveyors estimate that the resident known locally as the Mole Man has scooped 100 cubic metres of earth from beneath the roads and houses that surround his 20-room property.

"I often used to joke that I expect him to come tunnelling up through the kitchen floor," said Marc Beishon, who lives a few yards from Mr Lyttle's house.

His wife, Joy, sees the serious side of the issue, however. "We moved in six years ago and we've been complaining to the council ever since," she said. "Until six weeks ago they had the audacity to tell us the house was structurally sound. The whole of the opposite street lost power one day after he tapped into a 450-volt cable."

Now, after 40 years of complaints, the council has admitted Mr Lyttle's quarrying has put the neighbourhood at risk. Last week it obtained a court order to temporarily evict him in order to enable engineers to fill the holes with cement, at an estimated cost of £100,000 - for which Mr Lyttle will be billed.

"There has been movement in the ground," Phillip Wilman, a council surveyor, told Thames magistrates court. "He's fortunate a London bus is not in his front garden. The property is dangerous and liable to flooding."

A spokesman for the council said yesterday it had since offered Mr Lyttle temporary accommodation at a nearby hotel and removed 40 tonnes of excavated gravel and junk from his back yard. Structural engineers will descend into the dark labyrinth to gauge the full extent of the damage later this week.

For his part, William Lyttle denied that he has burrowed under his neighbours' homes, although he admitted to more than 40 years of "home improvements" on his own land. He told the Guardian the council's efforts to prevent him from re-entering his property breached his human rights.

"I first tried to dig a wine cellar, and then the cellar doubled, and so on. But the idea that I dug tunnels under other people's houses is rubbish. I just have a big basement. It's gone down deep enough to hit the water table - that's the lowest you can go."

His face lights up when he relates stories about holes under the towpath on Regent's canal or secret underground train networks. "I once dug a little tunnel out into the road for the cameras. But that's it," he insisted. "Tunnelling is something that should be talked about without panicking." The metre-wide openings seen by the few people who have descended down the shaft in his back garden, he said, were shadows.

Many of those who live around 121 Mortimer road beg to differ. In 2001, the pavement in front of Mr Lyttle's house collapsed, yielding a wide gash in the road. "You could see all the tunnels sprawling out all over the place inside - it was crazy," said one woman who lives nearby. "We don't wish the man any harm," said William Legg, 76, a retired civil servant. "He's a hard-working man - he just doesn't use his energy in the right way. Everyone around here just wants to see the place made safe."

But no one in the community, even those who remember when Mr Lyttle first purchased the house in the 1960s, can answer the question on everyone's lips: why does he do it? In all his years of digging, though, Mr Lyttle has never offered a straight answer.

"I don't mind the title of inventor," he said. "Inventing things that don't work is a brilliant thing, you know. People are asking you what the big secret is. And you know what? There isn't one."