© Victor J. Blue / Bloomberg Hennes & Mauritz flagship store in New York.

As Western consumers hole themselves up at home and stores are shuttered, retailers are suspending and canceling clothing orders, threatening millions of factory jobs in Asia just as China shows signs of recovering from the worst of the coronavirus outbreak.

Among the first to be hit by the consumer shutdown in the West are suppliers to the world’s “fast-fashion” giants, like H&M owner Hennes & Mauritz AB. Their business models depend on being able to get orders from factory floors to retail outlets in a matter of weeks. They are now pausing or canceling factory orders, boding ill for Asian manufacturers of other, slower-moving consumer goods like cosmetics, smartphones and cars.

Associated British Foods PLC, which owns Primark, a retailer with stores across Europe, and Peacocks Stores Ltd., a U.K. retailer owned by EWM Group, have suspended or canceled orders, according to public statements and notices to suppliers viewed by The Wall Street Journal.

Mostafiz Uddin, who owns a Bangladeshi jeans manufacturer, said his factory made around 14,500 jeans for Peacocks only to receive a letter saying that the store wouldn’t accept the order.

Get news and analysis on politics, policy, national security and more, delivered right to your inbox

“If they don’t take the goods, it’s a big loss for me,” he said. “What will I do with this?”

Mr. Uddin said he is negotiating with the retailer to receive payment for the jeans. Factory owners are usually very reluctant to take legal action because they don’t want to alienate buyers.

Related video: How Coronavirus Rescue Package Differs From Financial Crisis Bailout

In a March 17 email to Mr. Uddin, Peacock described its move as an “extreme measure,” but said “no stock will be allowed to be delivered into this business.” He responded that this “should be a time of support and not turning backs on partnerships.” Peacocks didn’t respond to requests for comment about the status of its clothing orders.

Primark and H&M say they, too, are pausing new orders. H&M said it is also evaluating “potential changes on recently placed orders.” The company buys from around 1,400 factories globally, with clothing production concentrated in China and Bangladesh.

Ulrika Isaksson, an H&M spokeswoman, said “our long-term commitment to suppliers will remain intact, but in this extreme situation we need to respond fast.”

An ABF spokesperson said that “no company could be expected to shoulder financial losses of the scale that would arise without ending these garment orders.” The spokesperson said the company is paying for inventory that is already at sea in transit, and is in talks with suppliers to explore other forms of mitigation.

In an emotional video address to clothing brands, Rubana Huq, president of the Bangladesh Garment Manufacturers and Exporters Association, a trade group, called on retailers to pay for goods they had already ordered.

She said that if retailers don’t support Bangladeshi factories, 4.1 million Bangladeshi workers could lose their jobs. “This is a social chaos we cannot afford,” she said.

Retail experts say the slowdown looks to be significantly worse than the financial crisis of 2008, when brands suffered from reduced consumer spending but could offer cut-rate sales and other deals to entice frugal customers. Then many factories were kept busy with orders, though overall demand was dented.

Times are different now, said Sheng Lu, a professor of fashion and apparel at the University of Delaware, because shops and malls are closed in much of the U.S. and Europe. “This time is a business lockdown,” he said. “The trade impact will be much larger.”

Mr. Lu said a 10% decline in exports for a country like Bangladesh could lead to a 4% to 9% drop in employment there, based on an analysis he conducted that correlates export growth with employment, using historical data from the International Labour Organization and the World Trade Organization.

“All of this is bad news for factories in Asia,” said Achim Berg, global leader of the apparel, fashion and luxury group at McKinsey. “Many of the big players are focused on surviving,” he said of retailers.

Aung Myo Hein, the head of a Myanmar garment industry association, says the industry has already shed 10,000 jobs. He has cut workers’ hours at his own factories but has avoided layoffs for now, he said.

In Malaysia, factory owners were optimistic in February when they began receiving diverted orders from China amid that country’s nationwide shutdown. Now Western buyers are sending emails to cancel. “Most of us will be forced to close shop soon,” said Tan Thian Poh, president of the Federation of Malaysian Fashion, Textiles & Apparels, a trade group.

Ramiz Khalid-Islam, a young Bangladeshi factory owner who manufactures Hanesbrands Inc.’s Wonderbra, said he’s working through his last order and plans to shut down his factory on April 1 for five weeks after European retailers canceled orders.

Two weeks ago, he was profiled in The Wall Street Journal for a different problem. His factory was struggling to maintain production because its Chinese employees were quarantined, and it couldn’t access raw materials from China.

According to Bangladeshi law, he will have to pay his workers around half their standard salary along with housing and medical benefits when the factory is shut down. Mr. Khalid-Islam says he plans to forgo his own salary for a couple of months.

“We don’t know how we’re going to afford this,” he said.