Philippine President Rodrigo Duterte has acknowledged for the first time that extrajudicial killings occurred on his watch, a concession that could buttress two criminal complaints filed against him at the International Criminal Court.

In what was described as a rambling speech to government officials outside the presidential palace Thursday, Duterte conceded that his “only sin is the extrajudicial killings,” according to the New York Times. He did not elaborate on his meaning.

Although he has claimed to have personally killed at least three people in the past, Duterte has long denied responsibility for extrajudicial killings that occurred as part of his administration’s brutal war on drugs. Officials claim that casualties of the drug war died as a result of resisting arrest.

The president’s latest remarks — during which he also invited the military to oust him should they feel he is not doing a good job — followed a drop in approval ratings amid high inflation and the rising cost of basic goods. Domestic surveys show his drug war continues to enjoy popular support.

Read more: Duterte is Assassinating Opponents Under the Cover of the Drug War, Philippine Rights Groups Say

Agnes Callamard, the U.N. special rapporteur on extrajudicial executions, called the admission an “extraordinary statement” among many heard this week as the General Assembly has convened to discuss some of the world’s most pressing problems.

Philippine police say they have killed more than 4,800 people in anti-drug operations since July 1, 2016, when Duterte took office. But human rights groups claim the death toll exceeds 12,000. Citing the number of deaths police classify as “under investigation,” opposition senator Antonio Trillanes IV — who was arrested this week on Duterte’s order — alleges that almost 30,000 people have been killed.

Read more: The Senate Standoff: How Duterte’s Attempt to Arrest Another Critic Is Testing Philippine Democracy

The Hague-based court prosecutes cases of genocide, crimes against humanity or war crimes when a state’s own legal system is either unwilling or unable to do so. In February this year, chief prosecutor Fatou Bensouda launched a preliminary examination into alleged crimes committed in the context of Duterte’s war on drugs.

The following month, Duterte announced that the Philippines would withdraw from the Rome Statute, the treaty that established the ICC.

Neri Colmenares, a human rights lawyer who helped relatives of eight people killed in the drug war file cases with the ICC last month, tells TIME that Duterte’s latest remarks could “be used against him as evidence.”

“We’re glad he made that admission in another of his unguarded moments,” Colmenares said.

Get The Brief. Sign up to receive the top stories you need to know right now. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Write to Joseph Hincks at joseph.hincks@time.com.