Republicans on Capitol Hill want to be able to deliver on President Donald Trump's newly proposed budget, which includes spending on infrastructure, defense and a border wall, House Budget Committee Chairman Steve Womack told CNBC on Tuesday.

But paying for those priorities, on top of the new GOP tax law, would likely add to the federal budget deficit, said Womack.

"While there are going to be some near-term deficit increases, we think the long-term health of the American economy is going to be much better as a result," the Arkansas Republican said on "Squawk Box." "If we can achieve 3 percent growth, then we have achieved a great victory."

The president's $4.4 trillion budget, unveiled Monday, includes federal spending of $200 billion for infrastructure upgrades; $18 billion for Trump's border wall; and $716 billion for national defense. The funding blueprint also calls for cutting $237 billion from Medicare; rolling back Medicaid expansion; and gutting Obamacare.

Womack argued that a Republican willingness to add to budget deficits does not represent a shift in the party's message, which holds fiscal conservatism as a core principle.

"We're still the party that believes in a very strong national defense posture. National security is very important to a lot of us," he said. "Building the nation's infrastructure is a key to helping jump-start an economy that's been in some anemic growth pattern for the past several years."