There is still a welcome mat outside A Touch of Class, and its website promises visitors "sumptuous earthly delights". But gentleman callers will be disappointed. After nearly 40 years of serving the local community, Sydney's most infamous brothel has shut up shop.

Politicians, judges and the late media magnate Kerry Packer were, reportedly, among those who frequented "Toucha", as it was fondly known. Mr Packer, Australia's richest man for nearly two decades, once hired the entire venue, it was rumoured, so that his polo-playing friends could enjoy themselves with "some good, clean girls".

Now the brothel, which operated out of three Victorian terraces in an unassuming residential street, is up for auction. The manager, Peter Lazaris, blamed competition from a rapidly proliferating number of cheaper establishments. He declined to comment on whether a raid by tax investigators had also played a role.

Download the new Independent Premium app Sharing the full story, not just the headlines

While the fixtures and fittings – including Roman fountains, crushed velvet furnishings and themed rooms complete with silk sheets and ensuite spa baths – are not included in the package, the bricks and mortar come replete with decades of colourful history.

A Touch of Class has been a Sydney landmark since 1972, but enjoyed its heyday in the 1980s, under the watchful eye of its most flamboyant madame, Zara Powell. Mrs Powell, a former convent girl, wrote a book about it, after leaving to found her own establishment, Zazzi's. But to the relief, no doubt, of many a pillar of society, she betrayed few secrets. The brothel prided itself on its discreet image; according to the Sydney Morning Herald, the adjoining terraces were known as the Three Wise Monkeys. For several years, there was a large truck parked permanently outside, ensuring that coming and goings were protected from prying eyes, including people sitting in a popular beer garden directly opposite. Rowdy drinkers, it was said, would heckle patrons on approach.

The truck collected a legendary number of parking tickets, and was replaced in the 1990s by a high wooden fence, which the local council declared illegal. In recent years, a dark green wrought-iron screen has shielded the premises. The brothel, located in a semi-gentrified neighbourhood near the city centre, was one of many local establishments – a search on Google lists 41 others within two miles.

But none had the same cachet, nor the same notoriety, nor the same knack of getting themselves into the papers. When someone was caught behaving badly, or using their company credit card to pay for services, it always seemed to be at "Toucha".

The brothel, which could fetch more than A$6m (£2.3m), also featured in a 2006 Australian film, Candy, starring the late Heath Ledger. The brothel was popular with visiting sailors, but had been open only intermittently of late, and missed out on what other establishments say was one of their busiest periods: World Youth Day in July, when thousands of Catholic pilgrims descended on Sydney for a week-long celebration.