Apart from being Australia's second longest-serving governor-general and introducing the first version of Medicare, Bill Hayden is probably best known for being a vocal and even hostile atheist.

Despite this, Gough Whitlam once told him, "Comrade, the Catholic Church will get you in the end".

It seems he was right.

Much to everyone's surprise, Mr Hayden was baptised last month at the age of 85.

Mr Hayden was treasurer and minister for social security in the Whitlam Government, foreign minister under Bob Hawke and then appointed governor-general in 1989.

One aspect of that last appointment caused quite a stir — the atheist and one-time republican was the first governor-general to make an affirmation, rather than an oath on the Bible.

John Howard said appointing Mr Hayden governor-general was "a bit like appointing an atheist as Archbishop of Canterbury".

'I still have questions'

Mr Hayden used to be "straight out opposed to religion".

"My father was a very bitter anti-religionist. I think that got to me," he told 7.30.

Bill Hayden at his home in Queensland. ( ABC News: Amy Donaldson )

"I'd seen a film in Brisbane at the Regent theatre one day and it was a thing on the Vatican and they had all these gold figures — seraphim and cherubim.

"I said they should be dragging them out of the ceiling, melting them and distributing them amongst the poor to help them. They're just so super rich they don't care.

"It drove me mad."

His antipathy to the church was tested when he and his wife Dallas lost their eldest daughter Michaela in 1966.

She was just five when she was knocked down by a car as she ran across a road.

Despite the help of a kind priest, Mr Hayden said he could not find solace in prayer.

"I was worried if I did it would be dishonest. I had to be honest," he said.

"I got letters from people who'd say, 'don't worry, don't ask yourself questions why God would do this in his inscrutable manner, he knows best'.

"And I thought you don't want a God, you want a Moloch that devours infants.

"I think I was certifiably insane in that period. Dallas had much more strength than me."

But it was four years ago when things really took a turn.

Mr Hayden spent seven months in hospital after a stroke, which gave him a lot of time to think.

"Don't think I was an atheist just by chance. I thought a lot about it, and I still have questions," he said.

The inspiration of Sister Angela Mary

Perhaps the biggest inspiration for his conversion has been Sister Angela Mary Doyle, administrator of the Mater Hospital in Brisbane for 23 years.

Their friendship over 40 years has been a communion of shared values and often bitter public battles to defend them.

Bill Hayden and Sister Angela Mary in 1989 ( Supplied: Mater Hospital )

"I've known of him since 1973," Sister Angela Mary told 7.30.

"It was a time when the national insurance scheme, Medibank, was being introduced. It was Bill Hayden's responsibility to promote it and promulgate it.

"I saw a man that was very honourable.

"People who were against it said to me — I met them at hospital conferences — one of them looked at me and said, 'And I thought you had brains'. It was very bad."

Sister Angela Mary said she took on the fights beside Mr Hayden because "what's right is right".

"We did many things that could have done us harm, for instance in helping the people who had AIDS.

"The premier of the day said nobody should help them. But we could not neglect people in need in order to make our own position secure."

She said she thought Mr Hayden's baptism "was a natural outcome of the goodness that he had lived through in his life".

"What Bill did the other day was not to draw attention to himself but to remind people, I think, to remember God."

'I've always been here, I shouldn't have wandered off'

Mr Hayden says the baptism ceremony meant a lot to him.

"When I went into the church that day, it was a hot day outside, and inside was very cool. It felt like a sanctuary, and I felt elevated in my chest, it was sort of ethereal," he said.

Bill Hayden felt 'elevated' when he was baptised. ( ABC News )

"And I thought, 'I've always been here, I shouldn't have wandered off'.

"I do believe Jesus was such a magnificent man, he suffered for our shortcomings.