House Speaker Paul Ryan, R-Wis., praised outgoing White House chief of staff Reince Priebus upon the news that he has been replaced by Homeland Security Secretary John Kelly, saying that he "left it all out on the field" during his White House tenure.

"Reince Priebus has left it all out on the field, for our party and our country," Ryan said in a statement. "Here is a guy from Kenosha, Wisconsin who revitalized the Republican National Committee and became White House chief of staff. He has served the president and the American people capably and passionately. He has achieved so much, and he has done it all with class. I could not be more proud to call Reince a dear friend."

Ryan also congratulated Kelly, who will start as chief of staff for the president on Monday.

"I congratulate Secretary Kelly on his appointment, and look forward to working with him to advance our agenda," Ryan said.

Just over 24 hours ago, Ryan became one of the only high-profile Republicans to publicly back Priebus. He told reporters at his weekly press conference that he was doing a "fantastic job" as chief of staff.

"As you know, Reince is a very good friend of mine. Reince is doing a fantastic job at the White House, and I believe he has the president's confidence," Ryan said at the time.

Sen. Ron Johnson, R-Wis., also gave high marks to both Kelly and Priebus and said that he looks forward to working with Kelly moving forward, calling him an "outstanding leader."

"Reince Priebus is a good friend. I thank him for his service and wish him the best of luck going forward," Johnson said in a statement. "Secretary Kelly is an outstanding leader who has served his country honorably and I know he will do an excellent job at the White House. I look forward to working with the administration as they choose a new leader for the Department of Homeland Security."