Republican House Speaker Paul Ryan said Monday he is "extremely worried" about President Donald Trump's plan to implement new tariffs on steel and aluminum imports.

But Trump shot back not long afterward, telling reporters in the Oval Office he was not going to back down on tariffs.

He did, however, again signal that he may drop tariffs for Canada and Mexico if they can renegotiate a new NAFTA deal with the U.S. Trump also singled out China as the biggest problem the U.S. faces on trade — a notion he repeatedly expressed during his campaign.

Stocks rose Monday afternoon as traders saw an easing in concerns over a potential trade war. A Republican source told CNBC that congressional leaders have not ruled potential action if Trump follows through on his plan.

Ryan is trying to push the Trump administration not to slap 25 percent tariffs on steel and 10 percent tariffs on aluminum, a plan announced by the president on Thursday.

"We are extremely worried about the consequences of a trade war and are urging the White House to not advance with this plan," said AshLee Strong, Ryan's spokeswoman. "The new tax reform law has boosted the economy and we certainly don't want to jeopardize those gains."

Later, White House spokeswoman Sarah Huckabee Sanders said: "We don't have to agree on everything" with Ryan. Her remark echoed Trump's earlier insistence on his hard line.

"No, we're not backing down," the president said later during an Oval Office meeting with Israeli Prime Minister Benjamin Netanyahu. "We had a very bad deal with Mexico, we had a very bad deal with NAFTA."

Trump said the U.S. had been "ripped off" by other countries on trade. "We lost $800 billion on trade," he said. "We are going to take care of it."

But Trump left a little wiggle room on his tariff stance. "Right now," he told reporters, he is "100 percent" backing the tariffs – "but it could be a part of NAFTA," he added, referring to the ongoing negotiations with Mexico and Canada.

Officials from the U.S., Canada and Mexico met for the final round of NAFTA talks Monday, but sources told Reuters that the renegotiations would end without a trilateral statement between the countries.

— CNBC's Christina Wilkie and Mike Calia contributed to this report.