A controversial glass box structure that garnered praise and jeers, depending on the eye beholding it, has been ditched from redevelopment plans for The Rocks.

Also known as the "jewel box", or by the less flattering "ice cube on stilts", the building was part of a $30 million proposal by private hospitality company Tallawoladah to restore and redevelop the historic Campbell's Stores warehouses, which overlook the harbour in Circular Quay.

An artist's impression of the controversial proposed "glass box" building.

The glass box, intended to house high-end shopping, was to be built next to the 165-year-old warehouses, and was expected to be as high as the neighbouring Park Hyatt Hotel.

However, in response to extensive community backlash, including an unprecedented written objection by then-heritage minister Mark Speakman, the structure has been ditched from the final design.