Paul Ryan has publicly admonished Trump’s message three times now, but the speaker of the House says he still plans to support the eventual Republican nominee for president no matter who he is.

After a weekend of tension-packed Trump gatherings Ryan said he blames both liberals and Trump-rally attendees for escalating clashes at rallies, but he does think Trump needs to do more to discourage violence.

“It’s pretty clear that there is a concerted effort on the left for people to interrupt these rallies, to interrupt these events and that needs to be condemned and we condemn that,” Ryan said. “Having said that, I think as candidates, all candidates have an obligation to do what they can do to try to provide an atmosphere of harmony, to reduce violence, to not incite violence.”

In a tweet following the press conference, Ryan’s spokesman said that in his role as chairman of the Republican convention, Ryan is required to stay neutral.

Reminder: He’s chair of the convention in Cleveland. He has to stay neutral. He can’t preside over it any other way. https://t.co/RzfWuDMeZA — AshLee Strong (@AshLeeStrong) March 15, 2016

It is clear that Ryan is frustrated by Trump’s message, which he has distanced himself and his members from on several occasions. So far, Ryan has spoken out against Trump’s plan to block Muslims from entering the country. Ryan has also condemned Trump’s support from David Duke. But, Ryan argued Tuesday that there was little he could do outside of that to deter voters from backing Trump.

“What we can control is who we are and what we do here in the House,” Ryan said. “All these other things on the top of the ticket are things that are not within our control.”

When asked how he could still support Trump given their fundamental differences, Ryan said that it was not up to him to determine who voters elect.

“The Republican primary voters are going to make this decision. This is not our decision to make,” Ryan said. “I am the speaker so I am going to keep doing what I said I was going to do, which is when I see principles being distorted, I am going to stand up in defense of our principles.”