(CNN) A Philippines presidential candidate under fire for his comments on gang rape has disowned an apology issued by his party and vowed he would "never really apologize" for the remarks, CNN Philippines reports.

Pressure has been building on front-running presidential candidate and Davao City Mayor Rodrigo Duterte since he made a joke in connection with the rape and murder of an Australian missionary during a 1989 hostage crisis in Davao City during a campaign rally on April 12.

After days of balking at calls for him to say he was sorry for the comments, Duterte finally seemed to repent Tuesday, when a statement was issued by his political party, PDP-Laban, with an apparent apology, CNN Philippines reported.

"I apologize to the Filipino people for my recent remarks in a rally. There was no intention of disrespecting our women and those who have been victims of this horrible crime," read the statement.

But asked about the statement later in the day during an appearance in Iloilo City, the 71-year-old seemed to back away from it, according to CNN Philippines.

"Don't force the issue because I will never really apologize," he said, according to a CNN Philippines translation of his comments.

He said he knew nothing about the statement of apology released by his party, and said the offensive remarks had been uttered in a burst of anger at the criminals.

'Let it bring me down'

"I was angry she was raped, yes that was one thing. But she was so beautiful, I think the mayor should have been first. What a waste," he said, according to a CNN Philippines translation of the comments, which appeared in a YouTube video over the weekend.

Despite the widespread condemnation, Duterte has repeatedly defended the remarks, saying Monday night that it was simply how people like him, who were not born rich, spoke, according to CNN Philippines.

"If it brings me down, let it bring me down. If it brings me up to the presidency, then well and good. I will serve you, but I will not as a matter of honor apologize for (it)," he told reporters.

Australia: Don't trivialize rape, murder

The Australian embassy in the country was critical of his comments on Facebook: "Rape and murder should never be joked about or trivialized."

The remarks also drew heated criticism from his political rivals in the upcoming May 9 presidential election, which one recent poll tipped him to win.

Jejomar Binay, current vice president and presidential candidate, said that Duterte didn't deserve to be elected to the country's top job.

"Rape and murder of a woman is not funny at all. Mister Duterte, don't you have a mother? Don't you have daughters? Your statement is simply revolting."