1. The anchors of Edward Lombe

The surface beauty of Sydney Harbour is nowhere more dazzling on a calm day than just off Middle Head. Look east and you can see the last gasps of the sea rolling between the heads and into the harbour. Then turn your head and watch the water splinter into spume as it crashes into the feet of the sandstone bluff. It is a stunning experience.

Sign up to receive the top stories from Guardian Australia every morning

Yet just under the harbour’s beautiful skin are a couple of rusting reminders of how cruel water can be.

When I paddle this stretch of the harbour, I think about two anchors somewhere below me. They are all that remains of the Edward Lombe, a three-masted barque that was smashed on to the rocks at Middle Head during a gale in 1834. Twelve people were killed. The disaster horrified Sydneysiders. They saw passengers and crew desperately clinging to the shattered stern, they helped to rescue survivors, and they grieved for the dead. The authorities responded by improving navigational aids in the harbour. That is one legacy of the Edward Lombe: the two anchors are another.

An anchor believed to be from the wreck of the Edward Lombe, among soft corals and sponges off Middle Head. Photograph: D Nutley

The maritime archaeologist Tim Smith has shown me photos of the anchors resting on the harbour bottom. They look both haunting and beautiful. These reminders of a tragedy are now festooned with marine life, including soft corals. – Scott Bevan

2. The ironbark of St John’s

Trees preserve a different sense of time’s passing than the built environment. Outside of the Royal Botanic Garden, the city’s oldest tree is thought to be a grey ironbark that can be found in the grounds of St John’s Anglican church in Glebe.

Facebook Twitter Pinterest The ironbark tree of St John’s Anglican church, Glebe, New South Wales, pictured circa 1875. Photograph: City of Sydney

Dating from the mid 1800s, it is a remnant of the forest of turpentine and ironbark trees that once grew across the Sydney region. This tree gives us a way of thinking about the land that enriches its present, urban identity. It connects us to its deeper, ongoing identities – as Gadigal land, as a natural ecosystem, and as something that lives, grows and changes. – Vanessa Berry

3. Bantry Bay, Middle Harbour

In a city obsessed with water views, Bantry Bay remains a comparatively little-seen wonder. This gangly strip of water at the top of Middle Harbour is cradled by the bushland of Garigal national park. Along the bay’s western shores are historical magazines, once used to store explosives, but these days, Bantry Bay is a repository of peace and quiet.

Facebook Twitter Pinterest Overlooking the waters of Bantry Bay. Photograph: Scott Bevan

For me, Bantry Bay is filled not with saltwater but a magic potion that clears the head and refreshes the soul. Every time I kayak into the bay and sit among the mangroves, listening to falling water and birdsong, I find it hard to reconcile that the bay is less than 10km from the city centre.

But not even in Bantry Bay can you leave behind one Sydney obsession. On weekends and holidays, Garry the coffee boat man chugs around the bay in a converted US army boat the colour of a Vittoria can, servicing those who have sailed and cruised into here to cast off the trappings of big city life – except caffeine. – SB

4. Wonderland City

Facebook Twitter Pinterest Despite its rollercoaster, elephant rides and haunted house, the Wonderland City theme park in Tamarama was short-lived. Photograph: Atomic/Alamy Stock Photo

If you were looking out over Tamarama beach in 1906, your view of the ocean would have been interrupted by a rollercoaster. The track ran in a loop, twisting and turning from one side of the cliffs to another, elevated on tall metal stilts. It was part of Wonderland City, a theme park that offered a rollerskating rink, elephant rides and a haunted house, among its many attractions.

Wonderland City was short-lived, closing in 1911. It might seem a quirky historical footnote today, but novelty sites have been and continue to be an enduring part of the city’s identity, as places of escape, wonder and fantasy. – VB

5. Looking Glass Bay, Parramatta river

Facebook Twitter Pinterest The view across Looking Glass Bay along the northern bank of Parramatta river. Photograph: Rina Marcellino

On the headland overlooking Looking Glass Bay, along the northern bank of Parramatta river, is a sculpture depicting one man showing another a telescope.

The sculpture records a moment in 1788, less than three weeks after the first fleet dropped anchor in Sydney cove, when the governor, Arthur Phillip, was leading an expedition up the river in search of suitable farming land. The exploration party had stopped for breakfast here when a group of Wallumedegal men approached the British explorers. One man, in particular, was very curious, examining the boats and their contents.

Facebook Twitter Pinterest Rockend cottage was once the home of the poet and storyteller Banjo Paterson. Photograph: Robin Smith/Getty Images

“The governor gave this man a hatchet & a looking glass,” Lieutenant William Bradley wrote in his journal, “which, when he looked into, he looked immediately behind the glass to see if any person was there & then pointed to the glass and the shadows which he saw in the water, signifying they were similar.”

Two cultures’ views were being changed before their very eyes.

The sculpture is in the grounds of a 19th century cottage, Rockend, which houses a restaurant. It was once the home of a poet and storyteller who had a profound influence on how many Australians have seen this land and themselves.

Andrew Barton “Banjo” Paterson lived here as a teenager with his grandmother. Paterson would catch the ferry to school at Sydney Grammar, and he enjoyed fishing and paddling on Parramatta river.

Paterson’s imagination would later roam the Australian countryside in poems such as The Man from Snowy River and Clancy of the Overflow, and he helped create a country’s unofficial anthem in Waltzing Matilda. But it is interesting to think how the cadences he heard in “the song of the river”, to use Paterson’s own words, helped shape who and what he would become – and, through his writing, who and what we have become. – SB

6. Bankstown bunker

Facebook Twitter Pinterest The interiors of Bankstown bunker were destroyed by fire but the bunker itself remains

The many tongues of Lost in Books, the only bookstore in Fairfield Read more

Stories of the underground have a huge imaginative appeal: the idea of a hidden city lurking just below the surface. The Tank Stream drain and the abandoned railway tunnels around St James station are some of the central city’s most well-known underground places.

The suburbs have an underground mythology as well, with sites that include the Bankstown bunker. Built as a military operations room during the second world war, the bunker was a maze of rooms three stories deep. Forgotten for decades, it was rediscovered, basically untouched, in the 1970s. The interior was later destroyed by a fire but the bunker itself remains, inaccessible except to the most intrepid urban explorers, underneath a housing estate in the suburban streets of Bankstown. – VB

• Vanessa Berry and Scott Bevan are discussing Sydney’s secrets alongside Luke Slattery and Tom Wright at Sydney writers’ festival on 3 May