It’s well past midnight on the tiny Estonian island of Suur-Pakri and the sun is just setting, casting a pink hue over the sea. We’re camping close to the water’s edge – and nipping between the intense heat of our improvised sauna and the freezing waters of the Baltic. Kessu, our guide, has rigged up a heavy-duty tent with a hole for a chimney and we’ve collected firewood. With just the stars and six other travellers for company, it’s the perfect end to an energetic day’s sea-kayaking.

I’m here for a long weekend adventure, exploring by kayak Suur-Pakri (large Pakri) and Väike-Pakri (small Pakri), two wild islands 50km west of the capital Tallinn. Although Estonia celebrated 100 years of independence from Russia last year, it was reoccupied by the USSR (and briefly the Nazis) from 1940 to 1991. The islands were emptied of residents in 1940 and remained a closed military zone until the early 1990s – and now they are an almost uninhabited wilderness, teeming with bird life, rare plants and stark reminders of Estonia’s turbulent history.

Our expedition begins on the mainland at the tiny harbour of Kurkse. After packing camping gear and food into waterproof bags, Kessu warns us that while the water looks perfectly calm, the nearby open seas of the Gulf of Finland and western winds mean conditions can change very quickly.

We paddle out for 6km across the choppy Kurkse Straits against the wind. As we near Väike-Pakri we see swans and, in the distance, herons watching from rocks. Moving closer to the coast, we spy reddish cattle free-grazing on fields along the shore.

Camping on the beach on Suur-Pakri, with the makeshift sauna in the tent. Photograph: Richard Nelsson

After setting up camp behind a raised shingle beach on Suur-Pakri, it’s time to explore on foot. Both islands are roughly 6km long, 2.5km at their widest, and almost totally flat. They’re formed from a limestone plateau, and the thin layer of topsoil means much of the vegetation – known as alvar – is sparse grass, juniper bushes and scrub, rather than the pine forests found in much of Estonia.

A path through this austere but attractive landscape leads to a bridge linking the two islands. Crossing over, we pass through meadows, eventually coming to a farmstead at Suurküla, once one of five villages populated by Estonian Swedes.

By the mid-1930s, there were more than 300 people living and farming on the islands. After control of the islands was returned to Estonia in 1994, they were cleared of mines and military hardware, with some of the plots being returned to their pre-war owners. A few of their descendants even moved back. Now, a couple of families live on Väike-Pakri, farming and restoring the landscape after years of neglect. As no grazing took place during the occupation, the open landscape and ancient fields gradually became covered in bushes and trees, but regular military exercises prevented the islands becoming totally overgrown.

The new inhabitants also cater for occasional visitors like our kayak party. Dinner, served in a rough wooden shelter, is a delicious stew made from their Hereford and Highland beef, along with homegrown vegetables.

Sunken treasure … the wreck used for Soviet target practice. Photograph: Richard Nelsson

Later we wander back to the sauna. Our firewood search reveals craters, piles of parachute cord and old bomb casings – evidence of the island’s former use. After a sound night’s sleep and breakfast of muesli and coffee, we take to the water again.

After 20 minutes we spy a shipwreck jutting out of the water, its rusting metal hull twisted and riddled with shell holes. It’s not a casualty of some forgotten battle, but a boat deliberately sunk by the Soviet military for target practice during their post-war occupation, says Kessu.

Hard paddling takes us past low limestone cliffs on the north-east coast of Väike -Pakri. In places, slabs of rock have broken off, like playing cards sliding into the sea. Later, in the distance on the mainland, we can make out the former closed city of Paldiski, once a Soviet nuclear submarine base.

Needing a break from the waves, we head for a small beach on the island, from where a path leads up to an escarpment. Some of us climb up old military watchtowers and explore long-abandoned tunnels and bomb shelters. Further along the coast lies the site of another village, called Väikeküla: all that remains of it is a stone church bell-tower and a graveyard. Hanging on a wall, a photographic display shows just what a thriving community it had once been.

While previous experience of kayaking would be helpful for the trip, it is not essential, and full instruction is provided. We’ve all taken on board Kessu’s tips, such as using the knees for stability and leaning into the waves, and with everyone now feeling confident in their kayaks the paddle back to the mainland goes by quickly and we are soon on dry land and heading into Tallinn for the night.

Tallinn’s Teletorn tower houses an exhibition on Soviet-era Estonia and also offers 360-degree views of the city. Photograph: Getty Images

I stay on for a day after the group disperses, exploring the well-preserved medieval old town and hipster HQ Kalamaja district, with its markets and art galleries. But the Pakri visit has stirred an interest in Estonia’s postwar history, so I also check out the Teletorn (€13, child €7), a 20-minute bus journey from the centre. Built for the 1980 Olympics, this 314-metre-high concrete edifice houses a fascinating exhibition about life in 1970s Soviet-era Estonia.

The real highlight of Teletorn, though, is the 360-degree view. Looking out towards a glistening Baltic Sea, I strain my eyes searching for the Pakris. I’m already missing the stillness and strange beauty of that wilderness: limestone cliffs we glided past, the free-roaming red cattle and, most of all, the way nature is reclaiming this once heavily militarised zone, gradually erasing a dark period from the country’s past.

• The trip was provided by Much Better Adventures. A two-night trip costs £293, including accommodation, airport transfers, equipment, all meals but not flights

Looking for a holiday with a difference? Browse Guardian Holidays to see a range of fantastic trips