The next business to take over Sydney Cove Oyster Bar's prime Circular Quay position "would have a very hard act to follow", says former prime minister Tony Abbott, one of the renowned restaurant's most notable customers.

Members of Sydney's culinary scene are wondering who will take over the well-known brown-brick venue on the tourist promenade following the restaurant losing the tender for the site after more than three decades.

The Sydney Cove Oyster Bar has lots its tender for its iconic site. Credit:Kate Geraghty

The former Liberal leader, who frequented the restaurant, was among those to praise its success in the 32 years it has occupied a converted toilet block, describing it as a "bit of a Sydney landmark".

"Anyone who takes it over would have a very hard act to follow," Mr Abbott said.