The fell 0.35 percent to 2,444.04, with consumer discretionary and industrials leading decliners. The Nasdaq composite fell 0.3 percent to end at 6,278.41 as big tech stocks like Amazon and Facebook pulled back.

The Dow Jones industrial average fell 87.80 points to close at 21,812.09, with Boeing and Johnson & Johnson contributing the most to the losses. Shares of Wal-Mart rose earlier in the session after the company announced a partnership with Google to offer voice shopping .

U.S. stocks fell on Wednesday after President Donald Trump said he would be willing to let the government shut down if his border wall isn't funded.

"With so many uncertainties weighing on US equities due to their overvalued levels, these two threats are ones investors could have done without," said Komal Sri-Kumar, president of Sri-Kumar Global Strategies, in an email to CNBC.

President Trump also expressed doubts whether the United States can reach a deal to renegotiate the North Atlantic Free Trade Agreement, adding that the government will "probably" terminate the trade agreement.

At a rally in Arizona Tuesday night, Trump said: "If we have to close down our government, we're building that wall ."

Traders work on the floor of the New York Stock Exchange

The Mexican peso dropped 0.2 percent against the dollar after the president's comments on trade and the border wall.

The price of gold futures for December delivery rose $3.70 to settle at $1,294.70 per ounce after the president's shutdown comments.

Stocks climbed off their lows in the afternoon after House Speaker Paul Ryan praised Republicans' tax reform plan. Ryan characterized the plan as critical to simplifying the current tax code and allowing the U.S. economy to grow even faster.

Wednesday's decline follows the Dow's best day in four months. The index posted a gain of 196 points on Tuesday, rebounding from a spell of investor caution on bolstered hopes of federal tax reform.

Some traders said the market's pullback Wednesday had little to do with the president's comments. "Most of us don't pay any attention to Donald Trump anymore," said Tower Bridge Advisors President Maris Ogg. "Yesterday's rally had something to do with speculation on the future of tax reform."

"I think this is a reversal of yesterday's enthusiasm," said Jack Ablin, chief investment officer at BMO Private Bank.

"This is all about legislation. The problem with that is The New York Times reported Trump and Senate Majority Leader Mitch McConnell aren't on speaking terms," Ablin said. "Tax reform still seems pretty remote, but luckily that's not fully priced into the market."

The stock market has roared higher this year, with the S&P 500 up 9.5 percent for 2017 and the Dow sitting near its all-time high of around 22,000.

However, Jeff Zipper, managing director of investments at the Private Client Reserve of US Bank, said there are some pockets of weakness in the market, including small-cap stocks and transports. The Russell 2000 and the Dow 20 Transports are up just 0.9 percent and 0.8 percent for the year, respectively.

"In the short-term, we're vulnerable. But in the long-term, I think the bull market is still intact," Zipper said. "We're still on a buy-the-dip mode and believe we'll keep grinding higher into year end."

On the data front, new home sales fell 9.4 percent in July and hit a seven-month low. The SPDR S&P Homebuilders exchange-traded fund (XHB) fell 0.93 percent after the data were released.

Wall Street also is fixated with a meeting this week in Jackson Hole, Wyoming, where a coalition of central bankers are gathered for the Federal Reserve's annual Economic Policy Symposium.

Both Fed Chair Janet Yellen and European Central Bank President Mario Draghi are set to speak on Friday. Yellen will likely discuss weak inflation and the Fed's plan to rein in its balance sheet.

Strategists doubt that the Fed will initiate any major policy changes or increase rates with inflation making slow gains over the past few months. Market expectations for a rate hike in December are just 36 percent, according to the CME Group's FedWatch tool.