Story highlights Sen. Rand Paul said new House Speaker Paul Ryan's vote in favor of the budget deal starts the House-Senate relationship off "on a bad foot"

Paul's comment stood out because many congressional Republicans have chosen not to rip the new speaker over his vote

Washington (CNN) Sen. Rand Paul knocked new House Speaker Paul Ryan for his vote in favor of last week's budget deal, saying Tuesday that it got the relationship between the Senate and the House started "on a bad foot."

Photos: Paul Ryan, rising GOP star Photos: Paul Ryan, rising GOP star Rep. Paul Ryan, R.-Wisconsin, was elected the 54th speaker of the U.S. House of Representatives on Thursday, October 29, after receiving the votes of 236 members. The vote was largely a formality after House Republicans nominated him for the position on Wednesday, October 28. Hide Caption 1 of 25 Photos: Paul Ryan, rising GOP star Ryan announced Monday, January 12, that he would not run for president in 2016, preferring instead to focus on policy work as chairman of the House Ways and Means Committee. Ryan, the GOP's 2012 vice presidential nominee, has long been seen as a top contender for the presidency. Hide Caption 2 of 25 Photos: Paul Ryan, rising GOP star Ryan, center, speaks with Rep. Devin Nunes, R-California, before a House Ways and Means Committee meeting on March 12, 2014. Hide Caption 3 of 25 Photos: Paul Ryan, rising GOP star Ryan and his wife, Janna, arrive at a state dinner at the White House in honor of French President Francois Hollande on February 11, 2014 . Hide Caption 4 of 25 Photos: Paul Ryan, rising GOP star Willie Robertson of the reality TV series "Duck Dynasty" poses for a picture with Ryan and his wife, Janna, before President Obama delivers his State of the Union address on January 28, 2014. Hide Caption 5 of 25 Photos: Paul Ryan, rising GOP star Ryan speaks at the Conservative Political Action Conference in National Harbor, Maryland, on March 15, 2013. Hide Caption 6 of 25 Photos: Paul Ryan, rising GOP star Ryan greets supporters during a presidential campaign rally with Mitt Romney at The Square at Union Centre in West Chester, Ohio, on November 2, 2012. Hide Caption 7 of 25 Photos: Paul Ryan, rising GOP star Rep. Paul Ryan of Wisconsin speaks during a campagin stop at the Iowa State Fair in Des Moines on August 13, 2012. It was the newly minted GOP vice presidential candidate's first solo stop since becoming Romney's running mate. Hide Caption 8 of 25 Photos: Paul Ryan, rising GOP star Ryan speaks after Romney announced him as his running mate in Norfolk, Virginia, on August 11, 2012. Hide Caption 9 of 25 Photos: Paul Ryan, rising GOP star Romney introduced Ryan as his running mate in front of the USS Wisconsin. The seven-term congressman provides a strong contrast to the Obama administration on fiscal policy. Hide Caption 10 of 25 Photos: Paul Ryan, rising GOP star Romney jokes with Ryan in April 2012 during a pancake brunch at Bluemound Gardens in Milwaukee. Hide Caption 11 of 25 Photos: Paul Ryan, rising GOP star Ryan looks on as Romney greets people June 18, 2012, during a campaign event in Janesville, Wisconsin. Hide Caption 12 of 25 Photos: Paul Ryan, rising GOP star Ryan speaks while campaigning for Romney at a textile factory in Janesville, Wisconsin, on June 18, 2012. Hide Caption 13 of 25 Photos: Paul Ryan, rising GOP star Ryan, left, and Romney greet each other on stage April 3, 2012, during the primary night gathering at The Grain Exchange in Milwaukee. Hide Caption 14 of 25 Photos: Paul Ryan, rising GOP star Ryan introduces Romney at a town hall meeting in Milwaukee, Wisconsin, on April 2, 2012. Hide Caption 15 of 25 Photos: Paul Ryan, rising GOP star Ryan is introduced before speaking about the federal budget at Georgetown University on April 26, 2012. Hide Caption 16 of 25 Photos: Paul Ryan, rising GOP star Ryan holds a news conference in December 2011 in Washington to introduce a package of 10 legislative reforms designed to revamp the budget process. Hide Caption 17 of 25 Photos: Paul Ryan, rising GOP star Ryan listens as Ben Bernanke, chairman of the U.S. Federal Reserve, speaks at the Committee for a Responsible Federal Budget annual conference in Washington on June 14, 2011. Hide Caption 18 of 25 Photos: Paul Ryan, rising GOP star Ryan looks over papers as he waits for other House Republicans to arrive for a news conference in the Capitol Visitors Center in 2010. Hide Caption 19 of 25 Photos: Paul Ryan, rising GOP star Ryan speaks to the media in 2009 about President Barack Obama's 2010 budget proposal. Hide Caption 20 of 25 Photos: Paul Ryan, rising GOP star Ryan, left, and Sen. Judd Gregg of New Hampshire speak to reporters about the 2010 federal budget. Hide Caption 21 of 25 Photos: Paul Ryan, rising GOP star Then-Budget Committee Chairman John M. Spratt Jr., left, and ranking member Ryan listen to Federal Reserve Chairman Ben Bernanke testify during the House Budget hearing on the economy on January 17, 2008. Hide Caption 22 of 25 Photos: Paul Ryan, rising GOP star Ryan follows President George W. Bush off of Air Force One at General Mitchell International Airport - Air Reserve Station in Milwaukee on July 11, 2006. Hide Caption 23 of 25 Photos: Paul Ryan, rising GOP star Ryan speaks at a Cato Institute briefing on Medicare reform in the Rayburn House Office Building in Washington on July 22, 2003. Hide Caption 24 of 25 Photos: Paul Ryan, rising GOP star Speaker of the House Denis Hastert, left, administers the oath of office to Ryan at the beginning of his first term as representative of Wisconsin on January 6, 1999. Hide Caption 25 of 25

"I think overall, Republicans want us to hold the line on spending and increasing the debt. That's why I start out disappointed because increasing the spending and increasing spending caps is the wrong way to go," Paul told CNN Tuesday afternoon. "I haven't met a Republican outside the beltway who is for raising the debt ceiling and raising the spending caps. So I think we've started out on a bad foot."

Ryan met with Senate Republicans for the first time as speaker Tuesday and hashed out ways the two sides could push more conservative priorities to President Barack Obama's desk.

Paul's comment stood out because many congressional Republicans, even members of the conservative House Freedom Caucus, who have slammed the budget deal have chosen not to rip the new speaker over his vote for it.