Washington (CNN) Democrats are optimistically looking to November to regain control of the House of Representatives for the first time since 2011, but if that doesn't happen, one Democratic leader said, it will be time to clean house.

Rep. Jim Clyburn said Friday night that if Democrats fail to win enough seats this November to take the House majority, the party's whole leadership team should step aside and make room for new faces.

"If we're still in the minority" after Election Day, "all of us have got to go," Clyburn told Politico after his annual "Jim Clyburn's World Famous Fish Fry" in downtown Columbia, South Carolina.

Clyburn appeared at the fish fry alongside fellow Democratic Rep. Tim Ryan of Ohio, who gave a keynote address at the Blue Palmetto Dinner, a fundraiser for the South Carolina Democratic Party, the same night. Ryan is considered and up-and-coming leader in the party and has not denied rumors that he is considering a 2020 presidential run.

But Ryan is also known for challenging current House Minority Leader Nancy Pelosi for her job in November 2016 and saying that the entire leadership team, Clyburn included, should step aside.

In June of last year, Ryan also said that Pelosi is more toxic than President Donald Trump.

"The honest answer is in some areas of the country -- yes, she is," Ryan told CNN's Don Lemon. "I think that in certain areas, like in some of these special election districts, it doesn't benefit our candidates to be tied to her."

Clyburn, the assistant minority leader as well as the highest-ranking African-American in Congress, insists there is no bad blood between himself and his younger Democratic colleague. During the fish fry, Clyburn described Ryan as a "good friend" and someone he spends time with outside of the Capitol.