A key part of House Speaker Paul Ryan's tax plan is getting no love from the New York Federal Reserve.

Its economists criticized Ryan's proposed "border adjustment tax," arguing that it would raise prices for American consumers and businesses on both imports and goods made in the United States.

New York Fed economists Mary Amiti, Oleg Itskhoki and Jozef Konings made the argument in a blog post Friday.

A border adjustment tax would give tax breaks to companies that export products and take away tax breaks from companies that import products. The idea is to encourage companies to produce in the United States, and remove the incentive for firms to relocate jobs overseas solely for tax purposes.

The White House sent mixed signals about it on Thursday.

"It could lead to a lot more jobs in the United States," President Trump told Reuters in an interview, without fully endorsing the border adjustment tax.

But Treasury Secretary Steven Mnuchin wasn't so positive. He told told CNBC: "We think there's some interesting aspects of it. we think there's some concerns about it."

Related: White House sends mixed signals on U.S.-Mexico trade ties

Ryan's plan has a big catch: it only works if the U.S. dollar rises 20% in value very quickly.

New York Fed economists doubt that the dollar would rise solely because of the new tax plan. They argue that other countries could implement the same tax or retaliate against the United States in other ways on trade.

It's also unclear whether the World Trade Organization, which includes the United States, would approve it. All those factors could hold back the dollar from rising 20%.

Retailers across America have already thrown up caution signs on the border adjustment tax.

They import a lot of clothes, TVs and appliances that they sell for low prices. Taking away tax breaks for retailers would probably raise prices for Americans, the economists argue.

Exporters could face higher hurdles, too. If the dollar rises some, but not by the full 20%, American products would still become more expensive -- and therefore less attractive -- for foreign buyers.

Related: Mexican senator ready to hit Trump where it would hurt: Corn

"This will provide incentives for our trading partners to switch their demand away from U.S.-produced goods, resulting in lower U.S. export sales," the economists write.

There's one more possible unintended scenario: Supply chains stretch over borders and oceans these days. Many companies that sell products overseas also import parts for those products from abroad.

Those companies would be subject to both higher taxes on the production side and lower taxes on the sales side. The overall impact wouldn't change, according to the post.

Ryan's office did not respond to a request for comment.

--CNNMoney's Heather Long contributed to this report.