Friends and family of Harold Holt gathered at Cheviot Beach in the southern state of Victoria on Sunday to mark the 50th anniversary of his disappearance.

The intimate service paid tribute to the former prime minister's life and political achievements. But for many Australians, Holt will always be better known for his fateful swim on December 17, 1967.

On the afternoon he vanished, the 59-year-old leader waded into rough surf while his alleged mistress Marjorie Gillespie watched on from the shore. Witnesses later told police that Holt swam out to the waves and appeared to get caught in a current before he was engulfed by swirling water.

Despite an exhaustive search, his body was never recovered. A subsequent inquest concluded that the death was accidental: that he had likely drowned and his body been swept out to sea.

What happened to Holt?

The mysterious nature of Holt's death, just two years into his term in office, left Australia stunned. It also gave rise to a torrent of conspiracy theories.

The prime minister was an experienced swimmer, snorkeler and fisherman, who had claimed to know the cove at Cheviot "like the back of my hand."

Some theorists claimed he had committed suicide, or that his judgment was impaired because of opiates he was taking for shoulder pain. Others suggested he may have suffered a heart attack, been bitten by a shark, or that he faked his own death so that he could run off with a lover.

Harold Holt cultivated a close relationship with US President Lyndon B. Johnson

A Chinese spy?

There was also speculation about possible Cold War plots, in which Holt was assassinated by the CIA, or abducted by Chinese communists in a submarine. In his 1983 book The Prime Minister Was a Spy, British journalist Anthony Grey went so far as to allege that Holt worked as a foreign agent for Beijing and had been collected by a waiting Chinese submarine.

A number of letters received by the Australian government at the time of Holt's disappearance, and since released by the National Archives, also subscribed to this theory: "I think that Mr Holt was kidnapped and taken away by submarines, and is being brainwashed for political information," one letter said.

"Our enemies know that Mr Holt and president [Lyndon B] Johnson were close friends. A word to the wise is enough — and I shall leave the rest to you and our government," another person wrote.

Harold Holt often went spearfishing at the isolated beach where he went missing

Most of these theories have been firmly rejected by Holt's family. "If you look at the beach here and how shallow it is, the idea that a submarine could have got in here is absolute madness," grandson Robert Holt told local broadcaster Channel Nine.

"My grandmother said he didn't even like Chinese food. So, it's unlikely (he was) taken away by a Chinese submarine," he added. "Every year there are people who drown in Australian waters and we never find the body."

'Politically courageous'

Holt was elected the 17th prime minister of Australia by a landslide in 1966. During his brief, 680-day period in office, he continued the pro-US policies of his predecessor Robert Menzies by expanding Australia's troop deployment in Vietnam. He also promoted greater engagement with Asia and the Pacific, significantly boosted funding for the arts, and ordered the switch from Australian pounds to Australian dollars.

In 1967, he held a historic referendum that paved the way for Aboriginal Australians to be counted in the country's census.

The executive director of the Menzies Research Centre, Nick Cater, told the Australian Broadcasting Corporation that Holt "modernized the office" of prime minister by implementing reforms that were "timely, innovative and politically courageous."

Around 2,000 guests, including US President Johnson and British Prime Minister Harold Wilson, attended a memorial service for Holt in Melbourne on December 22, 1967.