Every body of water in this country has a compelling story behind it. Australian swimming spots tell this country’s social and political history. Indigenous Australians – whose First Nations far outnumber those of modern Europe – harvested and swam in these waters for 60,000 years, for longer than any other human civilisation on this planet. The Freedom Ride of 1965 – led by activist Charles Perkins – was staged to protest the fact that the local Moree pool in northern New South Wales banned Aboriginal (‘quarter caste, half-caste or full blood’) swimmers. Aboriginal protestors picked up children from the local mission, escorted the kids into the pools personally and not only changed the pools’ policy, but set the stage for the successful referendum on Indigenous rights two years later.

These places where we swim remind us of the values to which Australians aspire: egalitarianism and fairness. What form of leisure is more accessible and democratic than dipping into a shared stretch of beach or a suburban pool on a baking summer’s day? You don’t need money to access the best spots this country has to offer. - Benjamin Law

Little Waterloo Bay, Wilsons Promontory, Victoria

Wilsons Promontory is the southernmost point of the Australian mainland, a little appendage of land that extends far into Bass Strait towards Tasmania. Like God and Adam in the Sistine Chapel, Victoria and Tassie are locked in an eternal near-touch. The area looks and feels more like Tasmania’s east coast than the rolling dairy hills of Victoria’s South Gippsland. Granite mountains dwarf the neighbouring landscape, and fern-filled valleys lead to squeaky white beaches. The Prom is a masterpiece that people travel across the world to experience. Unlike the Sistine Chapel, however, you will be required to stay overnight and strip to (near) nakedness to properly enjoy it.

While the casual daytripper can have a great time visiting the Prom’s easily accessible western shores, such as Norman’s Beach and Picnic Bay, the best and most sheltered swims are all on the east coast. Even during frequent winter squalls, Little Waterloo Bay tends to be an oasis of calm.

It’s hard to get to (even for the prevailing westerly winds), which is one thing that makes it so special. The only way to arrive is by foot. It’s a multi-day backpacking trip that offers protected coves and camp sites as a daily reward. While Little Waterloo is our favourite place, it’s as much about the journey as the destination.

There are a couple of ways to get in, but we like to do a clockwise circuit from Telegraph Saddle. We start at the carpark and walk 10.2 kilometres east (mostly downhill) to Sealers Cove. It’s a large yellow-sand beach with a wide creek at the south end. There is a good camp site here, but we just stop for a quick dip and lunch before continuing a further 6.4 kilometres to Refuge Cove. This tight, heart-shaped bay often has a few sailboats moored in its calm waters. In front of the campground, granite slabs form ramps down to the turquoise ocean.

Take a dip early in the morning to see mist rising from the pool’s surface.

From Refuge Cove it’s about 3.5 kilometres to Kersops Peak, where we get the first views of Waterloo Bay’s flawless white sand. The large sweep of beach is subdivided into smaller bays, separated by small headlands of temperate rainforest and granite boulders. Little Waterloo is among these private bays, only a few kilometres beyond the lookout.

From here you can continue further south to the very bottom of the mainlaind, but we only walk about a kilometre further before turning inland. An intermittent boardwalk leads 4.7 kilometres to Telegraph Junction and then a service road takes us 6.1 kilometres back to the carpark. Hot showers at Tidal River offer some consolation as we transition out of our Eden.

Dalhousie Springs, Witjira, South Australia

Located in Witjira National Park on the western fringe of the Simpson Desert, Dalhousie Springs is probably one of the most remote places that any of us will ever swim (or drive to). This group of 60 springs represents only a tiny fraction of the Great Artesian Basin, the world’s largest and deepest artesian system. Water at this location has likely travelled thousands of kilometres over about a million years to reach the surface, which certainly puts any road trip into perspective. After hours and maybe days of driving through the hot, dry desert on corrugated, unsealed roads, the thermal pool makes for welcome respite.

Main Pool is a large thermal spring and the only designated swimming hole in the area. Though it is surrounded by dense, wetland vegetation, it can be easily accessed via stairs on the water’s edge. The temperature sits at a reliable 38°C year-round, making it hospitable for all – most notably the six species of native fish, five of which are endemic to these springs. If you sit still long enough they will give you the full spa treatment, nibbling dead skin from any exposed part of your body.

When the warm spring water is well noodled, the expression ‘people soup’ comes to mind, but it’s also common to find that you have the pool all to yourself. However long you stay and whatever path you’re taking, you’ll emerge from the water feeling rejuvenated and ready for the next leg of your journey.

Greens Pool & Elephant Cove, William Bay, Western Australia

When people ask us about our favourite place to swim, our minds start racing through soggy, sweaty, sun-soaked memories, which almost always land at Greens Pool. Located on the edge of William Bay National Park and just outside the coastal town of Denmark, Greens is the community’s public pool. School swimming classes, bronze medallion training and games of Marco Polo happen here – it’s a beloved natural asset.

Giant granite stones act as a protective boundary around this sprawling, reefy pool, which is almost entirely closed off from the wild oceanic waters beyond. A tight-knit, motley crew of swimmers do a 500-metre lap around the perimeter, along the rocks each morning.

Locals know about keeping warm in cold water. Thermal caps protect you from an ice-cream headache while swimming.

Because of these long laps, it’s no coincidence that some world-class endurance swimmers train here. We meet a roster of regulars, varying in age and ability, many of whom swim year-round. They speak with great familiarity about every nook and cranny in the pool and talk proudly of the warm Leeuwin Current that moves down the west coast, allowing them to swim here in winter. But even in November, some are wearing wetsuits and thermals caps. There’s no doubt that this spot is completely weather-dependent, and on some days only for the boldest.

Just over the hill from Greens Pool is Elephant Rocks, a sheltered cove with granite boulders 20-times bigger than their namesake. Another glorious white-sand beach, the water here is clear and icy, and often all you can hear is the shrieking of other swimmers as they jump off the backs of the smaller ‘elephants’. On calm days, when the tide is high and the swell is mild, you can swim between Greens and the cove. Or you can make the short journey over the headland, peering down from the backs of the herd.

Arthur Bay, Magnetic Island, Queensland

We love finding places that are easy to access, yet still retain their wild beauty. Magnetic Island perfectly meets the brief. A 25-minute ferry from Townsville will land you on this relaxed, mountainous and secluded island. What a relief to be in the ‘dry tropics’. Sunshine and 30C is what most people expect around here, and you are pretty much guaranteed to get it.

A single road winds its way along the perimeter of the island, with regular buses shuttling people between the four municipalities – Picnic Bay, Nelly Bay, Arcadia and Horseshoe Bay. It’s easy to jump on and off as you please, getting close to your desired beach. There are 23 in total, but most can only be accessed by foot or boat.

Despite the island being such a popular destination, there are so many spots to swim that you rarely share a beach with a more than a handful of people. A short walk took us through open eucalyptus forests, past grunting koalas, down to Arthur Bay. To set the scene, the bay is bounded on both sides by granite headlands, where tall hoop pines find their footing among rounded boulders. A semi-permanent creek makes a natural wading pool at the back of the bay, shaded by pandanus trees. The main attraction, however, is the intense blue-green water of the beach. We were stinging for it by the time we reached the shore and our clothes have never come off so quickly. Underwater visibility is excellent here and we spent much of the day swimming around to the fringing reef on the north end of the bay.

Magnetic Island is surrounded by several shipwrecks, making it a popular place to dive. Sites at Florence Bay, Arthur Bay, Alma Bay, Geoffrey Bay and Nelly Bay attract large schools of angelfish, sergeant majors, parrot fish, batfish and trevally.

Stepping back onto the sand, thoroughly contented, we saw an alternate life path unfold in front of us. In this world, we build a driftwood shack and live off the sea. Our long, sun-bleached hair tumbles in tight ringlets around our brown shoulders. Strong, sinewy, sun-spot-covered arms pull in the catch of the day. We live easily, nourished by abundant marine-turtle tails and koala jerky. It’s a faultless, simple life until one of us succumbs to an entirely treatable infection. At this point, the reflection shimmers and we return to reality. Maybe we are better off just enjoying it as visitors.

This is an edited extract from Places we Swim by Caroline Clements and Dillon Seitchik-Reardon (Hardie Grant, $39.99)

Travelling around Australia? To find discount codes for local hotels, visit discountcode.theguardian.com/au