Australia's "first bloke" has landed himself in hot water over comments that men wanting to get their prostates checked should seek out small, female Asian doctors to perform the examination.

Tim Mathieson, the long-term partner of the prime minister, Julia Gillard, made the comments at a reception for the West Indies cricket team at the prime minister's official residence in Canberra, with Gillard standing right behind him.

One commentator observed that her face "performed a remarkable transition – jolly, carefree smile to impenetrable stone on the turn of a phrase".

Gillard – Australia's first female prime minister – became famous around the world last October for her fiery attack on sexism and misogyny in a speech to parliament directed at the country's conservative opposition leader, Tony Abbott. The speech has been viewed more than 2m times on YouTube and sparked a heated debate about the treatment of women in Australia.

Now, it seems, her partner has blundered into an issue in the national spotlight.

"We can get a blood test for it, but the digital examination is the only true way to get a correct reading on your prostate so make sure you go and do that, and perhaps look for a small Asian female doctor is probably the best way," Mathieson said to laughter across the room.

The national affairs editor of the Melbourne Age newspaper, Tony Wright, was scathing: "Uh-oh. In three words, he'd contravened an expansive sweep of the proposed anti-discrimination decrees. Small (sizeist, you might think); female (sexist); Asian (racist). We won't even go near digital penetration." Wright added that it wasn't Mathieson's first offence with badly chosen language.

Wright recalled that when meeting the Queen for just the second time in Buckingham Palace in 2011, Mathieson greeted her "with the immortal words 'You again!'", which was subsequently recounted across the pages of Australian newspapers.

Social media called for Mathieson, who has been a government men's health ambassador, to apologise for his comments on prostate cancer, which he said was supposed to be a joke.

"On reflection I accept is was in poor taste. I apologise for any offence caused," he said in a statement.

Gillard also criticised her partner's choice of words.

"Obviously, there's various ways of getting that message across, but he's certainly acknowledged that the joke cracked last night was in poor taste," she told ABC radio.

The Prostate Cancer Foundation of Australia downplayed the gaffe and said Mathieson had been a strong advocate for men's heath.

"Men often use humour to deal with difficult or embarrassing issues like the digital rectal examination, so whilst the joke was in poor taste, no doubt Mr Mathieson intended it as a lighthearted way of getting across the message that men need to be aware of their prostate health," the foundation said in a statement.

Prostate cancer kills 3,000 men each year in Australia.