"I think the market's a little bit more focused on the Fed moving on rates and the pace going forward. ... Additional rate hikes may be coming and the economy really isn't that strong but for some reason the Fed thinks it is, all because of the jobs number," said Robert Pavlik, chief market strategist at Boston Private Wealth.

Investors also awaited Friday's nonfarm payrolls report and weighed disappointment over the European Central Bank's decision, which sent the euro surging against the dollar for its best day in nearly seven years.

U.S. stocks closed sharply lower Thursday as yields climbed and traders were worried the Fed would raise rates while the economy is too weak. ( Tweet This )

"Super Mario stimulus disappointed and euro currency swings is the story. That's helpful to the Federal Reserve's case for going ahead and raising rates," said Doug Cote, chief market strategist at Voya Investment Management. "I think the market likes stimulus."

The Nasdaq composite closed about 1.7 percent lower after earlier declining more than 2 percent. Apple also came off session lows to end 0.95 percent lower.

The Dow closed down about 250 points after earlier falling as much as 304 points. The S&P and Dow both closed below their 200-day moving averages for the first time since Nov. 17.

A sharp rise in Treasury yields put pressure on interest-rate sensitive stocks. Financials was the third-greatest decliner in the S&P, while Goldman Sachs contributed the most to declines in the Dow Jones industrial average.

The S&P 500 closed down about 1.4 percent, falling back into negative territory for the year. Health care and energy declined more than 2 percent to lead all 10 sectors lower.

"I think a little part of it is just the unwinding of the kind of carry trade that was going on with the euro probably taking a little froth out of the market there. It's related to expectations the ECB didn't do a little more," said Andrew Burkly, head of institutional portfolio strategy at Oppenheimer Asset Management.

The U.S. dollar index fell more than 2 percent to below 98 after hitting a plus-12-year high of 100.51 Wednesday. The euro extended earlier gains to climb above $1.09 to hit its highest since Nov. 3.



The dollar index posted its worst daily loss since March 2009, while the euro had its biggest gain since that month.

The euro began rising ahead of the ECB decision after an inaccurate FT.com report said the central bank unexpectedly kept interest rates unchanged. The Financial Times later issued a correction.



"I think you've got an equity market that's trying to recalibrate to ... currencies and bonds that are moving more than they usually do," said Art Hogan, chief market strategist at Wunderlich Securities.

Read MoreNot so easy: Super Mario delivers shocking pain

Traders also said markets were still on edge over the latest news surrounding Wednesday's shooting in San Bernardino, California. The motive in the killings was unclear.



Treasury yields held near session highs in the close, with the 10-year yield around 2.32 percent but the at 0.95 percent after coming near 0.99 percent. The 2-year yield briefly fell back to 0.94 percent shortly before the stock market close.

"The market is responding to what Janet Yellen is saying and (her) being more optimistic about growth," said Patrick Maldari, senior fixed-income investment specialist on the North American Fixed Income team at Aberdeen Asset Management.

U.S. stock index futures dipped in pre-market trade after ECB President Mario Draghi said the central bank would extended its asset purchase program to at least March 2017 but did not increase the amount, falling short of expectations.

"There doesn't seem to be a change in the amount of monthly purchases outside of the money that will be reinvested. As I and some suspected, the dollar is a sell on the news and the selling right now is sharp as the details of more QE are out are not the big bang that some had though and/or hoped for," Peter Boockvar, chief market analyst at The Lindsey Group said in a note.



The ECB also cut its deposit rate to negative 0.30 percent from negative 0.20 percent, while leaving the refinancing rate unchanged at 0.05 percent.



"Investors were really rallying around the ECB and the Bank of Japan to continue easing and today's press conference was a disappointment in that regard. We had pretty high expectations," said Jack Ablin, chief investment officer at BMO Private Bank.



"We heard loud and clear from Yellen Wednesday on her intent to embark on tightening," he said. "The next big thing is really Friday, tomorrow, on the jobs number."



The German 10-year yield jumped above 0.6 percent. European stocks gave up gains to close sharply lower, with the German DAX off about 3.5 percent, on track for its worst week since August.



"It's almost as if Yellen and Draghi were coordinated. ... Not only did you have the compounding of the same message you also have an effect where European government yields have risen significantly. That gives us more room for U.S. interest rates to rise," said Brett Wander, chief investment officer, fixed income, at Charles Schwab Investment Management.

In a testimony before Congress Thursday morning, Fed Chair Janet Yellen reiterated confidence in the U.S. economic outlook. Her remarks followed a speech Wednesday that reaffirmed the case for the first interest rate hike in nine years as soon as this month, if the data is supportive. Her comments on Wednesday briefly sent the U.S. dollar to its highest since April 2003.



Qunicy Krosby, market strategist at Prudential Financial, said ever since Yellen indicated liftoff could come this year the market has been nervous and interpreting bad news as good news that could keep the Fed at bay.

"This tug of war means there is no direction in the market. Tomorrow it could all change. That's a market trying to decipher where we are headed but the road is going to be choppy. ... The deeper we go in December, the volume in the market tends to dissipate, which means the swings can be exaggerated," Krosby said.

Analysts said Thursday's dramatic moves in parts of the market could reverse after the jobs report Friday.

In U.S. economic news, , down from 59.1 in October.



Domestic factory orders rose 1.5 percent in October, beating expectations of 1.3 percent. Initial jobless claims rose to 269,000.

"I think what we're seeing is it's being clear the Fed is almost certain to raise rates in a few weeks andstocks don't like that. ... Also, the (services) ISM number was a little weaker than expected," said PNC Senior Economist Gus Faucher, noting traders were also awaiting Friday's jobs report.

Key for the week is November's nonfarm payrolls report, due Friday before the opening bell. The data is the final jobs report out before the Fed's December meeting.

U.S. oil settled up $1.14, or 2.85 percent, at $41.08 a barrel, helped by a weaker dollar and some speculation ahead of Friday's OPEC meeting. Brent briefly traded more than 4 percent higher above $44 a barrel.



Check out CNBC's special report on energy ahead of OPEC

U.S. stocks closed lower Wednesday after oil broke $40 a barrel and the Fed's Beige Book indicated more evidence of a weak manufacturing sector. News of the shooting in California during Wednesday's afternoon session also had traders on edge.

Yellen told a committee of lawmakers on Capitol Hill Thursday she does not see any impact on the U.S. economy from mass shootings or recent attacks around the world, but added they could have a knock-on effect in the future.

Read MoreWhy Thursday could help decide the course for markets into 2016



Separately, Fed Vice-Chairman Stanley Fischer said the Federal Reserve is developing regulations that would set minimum margins for securities financing transactions as it looks to contain risk and increase understanding of nonbank financial institutions.