Former Test batsman Greg Ritchie has defended himself against claims of racist and anti-Muslim comments at a Brisbane Test luncheon, refusing to apologise and saying: "If they take offence that's their choice."

The ex-Australian player and television personality was attacked for using the forbidden "k-word" during a lunchtime speech at the Brisbane Cricket Ground Trust members dining room on day one of the first Test against South Africa on Friday. He also made disparaging remarks about Islamic people in the address, South Africa's Sunday Times reported.

Greg Ritchie during his playing days on the 1985 Ashes tour. Credit:Getty Images

"I've got nothing against the Muslim people," Ritchie was reported as saying on Friday. "Just this morning I had to try and stop three little Muslim boys trying to break the lock on my car boot. "I had to say, 'Shut up! You're in there for a reason!'"

On Sunday, Ritchie told Fairfax: "That's a joke that I use, and I'll continue to use it. It's just a little humourous joke to indicate that they're not my favourite people of my choice. If they take offence that's their choice."