There are also Fed speakers late in the day. Minneapolis Fed President Neel Kashkari speaks at 3:15 p.m., and Philadelphia Fed President Patrick Harker will be before a group in New York at 5 p.m.

Besides the budget Tuesday, new home sales are expected at 10 a.m. ET, and Markit services PMI, a reading on service sector activity is released at 9:45 a.m. Most earnings are now out of the way, but Toll Brothers and AutoZone will be reporting ahead of the opening bell.

"The start of the budget process is extremely important because the fiscal year 2018 budget will include the reconciliation instructions for tax reform, assuming they're going to finish tax reform," said Daniel Clifton, head of policy research at Strategas. "If you don't pass a fiscal 2018 budget, there is no tax reform. This starts the process officially."

Markets may shrug off President Donald Trump's budget when it's released Tuesday, but it is an important starting point for something the market cares deeply about — tax reform.

Traders are also watching President Trump's overseas trip to the Middle East and Europe. Trump visited Saudi Arabia over the weekend and was in Tel Aviv on Monday where he said his visit to Israel was a rare opportunity to "bring security and stability and peace to this region and to its people."

"This is just standard fare. When things are going poorly at home, presidents seem to book it overseas," said McCarthy. Trump heads to Europe, where he will attend the G-7 leaders meeting at the end of the week.

Clifton said it's unusual for a president to be traveling overseas when their budget is introduced. The budget is usually something the president spends time promoting.

The budget proposes slashing federal spending by $3.6 trillion over the next decade through steep cuts across the government. Half the money would come form changes to such programs as Medicaid, food stamps and federal student loans. Deep double digit cuts are proposed for the next fiscal year from the Environmental Protection Agency, the State Department and the Education Department.

The Trump administration also proposes selling half the country's oil stockpile, from the U.S. Strategic Petroleum Reserve. That weighed on oil prices lower, as traders considered a huge gusher of new U.S. oil hitting the market as OPEC attempts to limit supply to balance the market. West Texas Intermediate crude futures were slightly lower, but still above $51 per barrel.

Defense spending would rise by $25 billion in fiscal 2018, one of the few areas where spending would increase.

"It's the beginning of the process and everyone knows this is a place holder," Clifton said.

Stocks closed higher Monday, as traders looked past the controversies surrounding Trump at home and bid up defense stocks and other stocks benefiting from deals with Saudi Arabia. Stock futures were higher Tuesday morning.

"He's having a good trip so far. He's stayed on script," said Bob Sinche, chief global strategist at Amherst Pierpont. Sinche said Trump could start to win back some confidence if his meetings overseas go well.

Trump's troubles have hit the dollar, at the same time the euro is surging on an improved view of the euro zone. The dollar index is now down 0.9 percent since the election, after being up more than 6 percent in early January. But the air needs to clear around the investigation into Trump's campaign's ties to Russia before the dollar takes off again, Sinche said. He said it could also be helped by the Fed raising interest rates, but it would really improve if it seemed that tax reform was becoming a priority in Washington.

Former FBI director James Comey is expected to testify before Congress on the investigation after the Memorial Day holiday, and some strategists say the dollar could be stuck until then. Comey reportedly has said Trump asked him to end the investigation into Michael Flynn, Trump's former national security advisor. On Monday evening, the Washington Post quoted sources saying that Trump asked two top intelligence officials in March to help him push back against the FBI investigation.

The stock market has held up despite a sell-off last week, and Clifton said there could still be policy positives even with the investigation. "We think the Comey stuff is serious. But at the end of the day, I can't be on the sidelines if a $200 billion infrastructure plan is going to be put on the table. That could be tomorrow, it could be in two weeks," said Clifton. The strategist also said Republicans could be galvanized to push tax reform because of the White House's troubles, opposite of what many in the markets think.

Watch: What's at stake in Trump's visit to Israel