“Stay at least 200m away from the whaling station – it’s filled with asbestos and the roofs could literally blow off,” warned expedition leader Nate Small, as we stepped gingerly out of our Zodiac and into the fizzing surf at Stromness Bay, South Georgia. I picked a careful route across the grey-pebble beach, eyeing warily the growling fur seals and slumbering elephant seals, their gargantuan bodies emitting a series of burps, bellows and rumbling bass notes.

At the far end of the bay, set against a mountain slope and surrounded by bog land, was a cluster of dilapidated, rusty, corrugated iron buildings. Huge sections of the roofs and walls were missing, and those that remained rattled incessantly in the near gale-force wind. It looked as if a natural disaster had struck. I stopped at an ‘Asbestos – Keep Out’ sign and peered through the encroaching mist, my extremities numb from the sub-zero conditions. It was a struggle to picture the station as a thriving community, yet a century ago Stromness was part of a highly profitable – and brutal – industry that transformed South Georgia into the whaling capital of the South Atlantic.

You may also be interested in:

• The city that lit the world

• A strange life at the end of the world

• Is this the world’s last paradise?

Earlier in my trip, Seb Coulthard, expedition guide and on-board historian for Polar Latitudes, told me how Ernest Shackleton arrived in Stromness in 1916 following his epic 1,300km escape from Elephant Island, one of the South Shetland Islands that lie just north of the Antarctic Peninsula, after his ship was trapped and later crushed by pack ice. For the polar explorer, the whaling station represented civilisation, but today nature is slowly reclaiming it. Fur seals sheltered beside a blubber cooker, king penguins waddled past disintegrating warehouses and skuas (aggressive, dark-brown seabirds) washed themselves in meandering streams that once ran with the blood of tens of thousands of whales.

A rugged, inhospitable land of glaciers, mountains and fjords, South Georgia is one of the most remote places on Earth. This sub-Antarctic British overseas territory in the South Atlantic is around 1,400km from its nearest inhabited neighbour, the Falkland Islands, and is only accessible by sea. Like me, the majority of the nearly 18,000 people who visit each year are on Antarctic cruises. The island spans 3,755 sq km – less than a fifth of the size of Wales – and around half is covered permanently by ice (though, as a result of climate change, its glaciers are drastically retreating).

Despite its isolation and harsh environment, South Georgia was once a vital part of the global economy. First sighted in 1675, this uninhabited island was claimed for Great Britain by James Cook in 1775. His accounts of abundant seal populations aroused the interest of sealers from the UK and the US. In little more than a century, South Georgia’s fur seals were hunted to the verge of extinction. By the early 1900s, sealing was no longer economically viable, but it was quickly replaced by an equally bloody industry.

The day after my visit to Stromness, my ship sailed south through 75-knot winds to King Edward Cove. Scattered with shipwrecks and mini icebergs, backed by forbidding mountains and obscured by drizzle, this sweeping bay was the location of South Georgia’s first whaling station, Grytviken. Today it is the site of the island’s main settlement, home to the majority of the 15 to 30 people, mostly scientists and government officials, who live on South Georgia at any one time.

After paying my respects to Shackleton, who is buried in Grytviken’s small cemetery, I was taken around the decaying whaling station by Finlay Raffle, a curator at the site’s museum. We walked through an industrial landscape of squat towers, warehouses, power plants, mazes of inter-connected pipes, and huge blubber and bone cookers, everything thickly covered with rust. Along the shoreline, ships and boats in varying stages of collapse were pushed up at odd angles by the tide. Chunks of whale bone carpeted the muddy ground.

In 1902, Norwegian polar explorer Carl Anton Larsen stopped in South Georgia and chanced upon a beautiful natural harbour. After the discovery of several sealers’ try-pots – used to render oil from blubber – the area was named Grytviken (‘Pot Cove’ in Norwegian). “They moored not far off from where your ship is today,” Raffle said. “The only difference was when they looked out over the water they saw hundreds of whales in this bay alone.” With the northern hemisphere whaling industry in decline due to the decimation of whale populations, Larsen spotted a business opportunity. He returned to Grytviken in November 1904 and set up a whaling station, which swiftly prospered. By 1912, there were six other whaling stations on South Georgia, including Stromness.

Narrowly dodging a pair of fur seals, who blended in remarkably well with the rusty machinery, we approached an old whale-catcher. With its steam-powered engine, reinforced hull and mighty harpoon gun, the whaling ship Petrel could capture as many as 14 whales on a single trip. Back at Grytviken, the animals would be winched onto a slipway, the ‘flensing plan’. “It was very slippery with all the blood and oil, so the men wore boots with nails in them to grip properly” Raffle said. “They had a flensing knife – a long, almost hockey stick with a sharp, curved blade, which they used to cut the blubber away.” The whole process took 20 minutes per whale.

Initially the whalers were only interested in the blubber, but later regulations forced them to use the whole of the carcass, Raffle explained, pointing out gory rotating blades and a 24-tonne blubber cooker. Although the meat and bone-meal were sold as animal feed and fertiliser, whale oil was the real prize. “The best oils went into food products like margarine and ice cream,” he said. “The second grade went into soap and cosmetics, and the worst was used in industrial processes.” Whale oil also provided glycerol, used in the manufacture of explosives, and high-quality lubricants for rifles, chronometers and other military equipment. As a result, demand soared during World War One and Two.

There were 450 men at Grytviken in its heyday, working 12-hour shifts, seven days a week, in temperatures that could plunge below -10C. Larsen was keen to look after their spiritual needs, building an impressive neo-Gothic church. But, said Raffle said, the pastor “was the least employed man on the station”. The cinema, windswept football pitch and ski jump – now just a few bits of broken timber protruding from a hillside – proved rather more popular. The community store, or ‘slop chest’, also provided distractions. “Tobacco was the most popular item but the men also bought lots of cologne,” Raffle said. “Larsen didn’t allow alcohol, so they drank cologne instead. They also had illicit stills, and even got boot polish, squeezed it through bread, and drank the drippings, which apparently also had alcohol. Anything to pass the time.”

Raffle left me at the former manager’s house, a simple, white-washed building that has been turned into the site’s museum. The displays inside contain some stark figures: 175,250 whales were processed on South Georgia between 1904 and 1965, when the industry collapsed due to over-hunting and developments in the petrochemicals industry. If you consider the Antarctic region as a whole and include the many ‘factory ships’ that processed whales on board, almost 1.5 million whales were killed between 1904 and 1978, when hunting of the species eventually ended.

Whale populations haven’t recovered. The International Whaling Commission (IWC) says blue whale numbers in the southern hemisphere have fallen from as many as 200,000 to the ‘low thousands’; fin whales have undergone a similar decline. There are an estimated 60,000 humpbacks in the southern hemisphere, but this is also far lower than the pre-whaling era. In September 2018, IWC plans for a South Atlantic whaling sanctuary were rejected by pro-whaling countries. Japan later announced it will resume commercial whaling for the first time in three decades, prompting global outrage.

It’s a bittersweet irony in that it was a terrible, brutal industry, yet nature took its sweet revenge by reclaiming it

The plight of the whales is undeniably bleak, but in other respects, South Georgia has become an improbable model of conservation. One of the world’s largest marine reserves, the South Georgia and South Sandwich Islands Marine Protected Area, was created here in 2012 to protect more than one million sq km of the surrounding waters, while seal numbers have bounced back: the island now has 98% of the world’s Antarctic fur seals and roughly 50% of its elephant seals.

South Georgia also has 30 million breeding pairs of seabirds. During my visit, I spent a morning at St Andrews Bay in the company of 400,000 king penguins – one of four penguin species found on the island – and an afternoon on Prion Island, an important breeding site for wandering albatrosses. Last year, South Georgia was declared rodent-free after a pioneering eradication programme, which the authorities hope will allow birds like the endemic South Georgia pipit and South Georgia pintail to flourish.

Despite the profusion of wildlife, it was the island’s whaling heritage that remained foremost in my mind as I sailed out of Grytviken. “When you walk about these stations all you see are these rusting boilers, blubber cookers and bone saws,” Coulthard said. “It’s a bittersweet irony in that it was a terrible, brutal industry, yet nature took its sweet revenge by reclaiming it. It’s a reminder that nature doesn’t need human beings; we need nature.”

This trip was made possible by Polar Latitudes. Trips to South Georgia are also available through Quark Expeditions, One Ocean Expeditions and National Geographic Expeditions, among other operators.

Join more than three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Culture, Capital and Travel, delivered to your inbox every Friday.