Western Australia's most iconic eyesore is finally being demolished after an epic battle between its fertiliser tycoon owners and the local council.

Pankaj and Radhika Oswal's $70 million partially built mansion dubbed the 'Taj on Swan', in the well-heeled Perth suburb of Peppermint Grove, was abandoned after the couple left Australia in December 2010 following the collapse of their ammonia company, Burrup Holdings.

The home was to include seven domes, a temple, gym, swimming pool and parking for 17 cars.

Pankaj (L) and Radhika Oswal have agreed to pay for the cost of the demolition. ( AAP: Julian Smith )

But the building site has deteriorated, with reports it had become a haven for squatters and drug addicts, and used to host illegal parties.

The council won its long-running battle to tear it down, after the Oswals failed to act on an undertaking in the State Administrative Tribunal to demolish it by September 30.

Peppermint Grove council is owed $108,000 in unpaid rates and the Australian Tax Office has a freeze on the sale of the property over millions in alleged unpaid taxes.

The couple will continue to own the 6,600-square-metre block, which sits on prime riverfront real estate in Perth's wealthiest suburb.

Michael Smith, a spokesman for the couple, told the ABC the Oswals had no problem with the demolition going ahead and they had people "liaising with the council".

Demolition teams moved in after a long-running battle over the future of the building. ( ABC News: Hayley Roman )

"They plan to sell the property, they will engage consultants to work out the best way to dispose of the land," he said.

"The contract cost of the demolition is $89,000 plus GST.

"The Oswals have agreed to pay this amount to the council."

It took just minutes for an iconic section of the partially built palace to be felled. ( ABC News: Hayley Roman )

'A dangerous eyesore'

Shire of Peppermint Grove president Rachel Thomas said the demolition would take about three weeks, but locals were happy it was finally happening.

"We are very, very glad that it is going. Apart from being an eyesore it is a dangerous structure," she said.

"It has been a terrible, terrible experience for the neighbours who have never known if they are going to be disturbed at night by kids having a party [in the abandoned site] and it has been a real nightmare for the council because the owners have done nothing, or very little, to secure the property."

The mansion was to include seven domes, a temple, a gym and a swimming pool. ( AAP: Tony McDonough )

The demolition attracted dozens of onlookers from neighbouring suburbs and afar.

Local residents and other interested onlookers gathered to watch the demolition. ( AAP Image: Tony McDonough )

Claremont resident Natalie Bevilacqua said she was looking forward to seeing the back of the partly built palace.

She said the building's architectural style and unfinished state were both an eyesore.

"It's just such a gorgeous family, lovely, well-established aadrea, [but] the [property] is just way out of keeping in tune with the neighbourhood," she said.

"I've known the kids to come here as a little spooky house to come into, it's almost been a little rite of passage to come and wander through the house after the sun goes down."

Inside the incomplete Taj on Swan in 2013. ( Supplied: Sean Guilbert )

Her daughter Tessa Bevilacqua was one of those who had explored the building in her youth.

"There's going to be some sadness when people realise their hideout is gone, but I think it's for the better," she said.

"It's such a large space, it's wasted with a house not being used. That area could be used for something like a park or more homes for people."

The Oswals are planning to sell the land the mansion was built upon. ( AAP: Lloyd Jones )

The ABC has requested an interview with the Oswals. Mr Smith said they were currently in Europe.

Last month, ANZ reached a commercial settlement with the Oswals after the couple sought $1.5 billion over the terms of sale of their ammonia empire by the bank.

The couple had argued the $US560 million sale of their 65 per cent stake in 2012 represented less than half its true value.