In his first sit-down interview with Donald Trump since he took office, Fox News Sunday host Chris Wallace got the president’s immediate reaction to the news that a federal judge had ruled in CNN’s favor, essentially forcing the White House to reinstate reporter Jim Acosta’s press access.

“It’s fine, it’s not a big deal,” Trump said at first.

But in his next breath, the president seemed to threaten Acosta, saying: “If he misbehaves, we’ll throw him out. Or we’ll stop the news conference.”

As part of his ruling, U.S. District Court Judge Timothy Kelly, who was appointed by Trump, said that the White House would need to establish rules of conduct for members of the press corps. Asked what those guidelines might entail, Trump said his staff is currently working on those “rules of decorum.”

“You can’t keep asking questions,” Trump said, indicating that Acosta was preventing other reporters from asking him questions at a recent press conference. However, the White House initially claimed they were revoking his access because of the way he pulled his microphone back from an intern who was trying to take it out of his hands, even going so far as to release a clearly doctored, conspiratorial video to defend its decision.

“Today, the court made it clear that there is no absolute First Amendment right to access the White House. In response to the Court, we will temporarily reinstate the reporter’s hard pass,” White House Press Secretary Sarah Huckabee Sanders said in a statement on Friday. “We will also further develop rules and processes to ensure fair and orderly press conferences in the future. There must be decorum at the White House.”

“Nobody believes in the First Amendment more than I do,” Trump, who has repeatedly referred to the free press as the “enemy of the American people” told Wallace. “And if I think somebody’s acting out of sorts I will leave. I will say, ‘Thank you very much everybody, thank you very much for coming’ and I will leave. And those reporters will not be too friendly to whoever it is that’s acting up.”