Story highlights Trump and Ryan held an hour-long call Saturday and another call Sunday

Trump's deepest frustrations appear directed at the House Freedom Caucus

(CNN) Not long after President Donald Trump told his Twitter followers to watch Fox News host Judge Jeanine Pirro's show Saturday night, the host and Trump ally took to the screen to deliver a harsh rebuke of House Speaker Paul Ryan.

"Paul Ryan needs to step down as speaker of the House," Pirro said. "The reason? He failed to deliver the votes on his health care bill. The one trumpeted to repeal and replace Obamacare. The one that he had seven years to work on. The one he hid under lock and key in the basement of Congress. The one that had to be pulled to prevent the embarrassment of not having enough votes to pass."

After the Republicans' monumental failure last week to repeal and replace Obamacare and hand Trump a key victory in his presidency's first 100 days, all eyes in Washington are on the relationship between the President and speaker, a political odd couple who seemingly forged an amiable alliance after a rocky presidential campaign, and whose unique skills -- Trump the ultimate closer and Ryan the policy guru -- seemed to complement one another.

Ryan went to the White House on Monday, and met with Trump, Ryan spokesperson AshLee Strong said.

On Sunday morning, White House chief of staff Reince Priebus told Chris Wallace on "Fox News Sunday" that the President's tweet to tune into a show that would skewer Ryan was merely "coincidental," and that Trump was trying to do his friend Pirro a favor, not insinuate Ryan should resign.

Read More