Popular Japanese fashion chain Uniqlo has a message for President Trump: We'll leave the U.S. if you insist we make all of our clothes in America.

Trump's main economic message is "Buy American, Hire American." He campaigned on reviving American manufacturing, especially in the Rust Belt.

But Tadashi Yanai, the billionaire head of Fast Retailing Co., which owns Uniqlo, doesn't like being given an ultimatum by Trump.

"If I was directly told to do so, I will withdraw from the United States," Yanai told Japanese newspaper The Asahi Shimbun this week when he was asked about Trump's Made in the U.S.A push.

Uniqlo currently has 51 stores in the U.S. Yanai said the company plans to open at least 20 more stores this year, although it's watching what Trump and Congress do on trade.

Yanai doesn't want to see a tax on foreign imports to the U.S . He argues any sort of tariff would raise costs and be bad for shoppers.

"We would not be able to make really good products [in the U.S.] at costs that are beneficial to customers," Yanai said in the Japanese news report. "It would become meaningless to do business in the U.S."

A Uniqlo spokesman added Friday that the retailer would "stay true to our core business values of offering the highest quality apparel at accessible prices."

As CNNMoney has pointed out before, American shoppers claim that they want to buy products made in the U.S. But in reality, they purchase things that are cheaper or perceived as better quality, regardless of where they were made.

Related: Trump is pushing 'Buy American.' But customers (mostly) don't care

Uniqlo is the latest foreign company to warn that there will be consequences from any moves to curb foreign trade.

And some experts worry that if Uniqlo and other companies pull out of America, that could make life even tougher for many struggling shopping malls.

Uniqlo and other fast-fashion chains like Forever 21, Zara and H&M are actually doing reasonably well while many traditional retailers such as Macy's, Kohl's and Sears struggle.

"Not only is Uniqlo a major retailer and employer in the U.S., it is also a major tenant of landlords in a landscape of retail distress," noted Peter Boockvar, chief market analyst at The Lindsey Group in Virginia in his morning note on Friday.