Washington (CNN) Kentucky GOP Sen. Rand Paul said Tuesday he wishes President Donald Trump had not moved to exit the Iran deal and supported talks between Trump and Iranian President Hassan Rouhani.

"I think that we need more discussion at a higher level with Iran," Paul said on CNN's "The Situation Room with Wolf Blitzer."

Trump announced last week he was removing the US from the nuclear agreement, and at a rally following his decision to exit the deal, Trump said he still hoped to make a better agreement.

Paul said Tuesday there's "no evidence" Iran violated the agreement forged during the Obama administration. He added that, while the deal had shortcomings, keeping it in place was preferable to "ripping it up."

"I think that it was a mistake to rip up the agreement, and we should've built on the agreement rather than stepping backwards," Paul said.

Read More