The following correction was printed in the Observer's For the record column, Sunday December 26 2010

The article below said incorrectly that an estimated 40 million workers in Bangladesh sew 1.5 million pairs of jeans every year. In fact, 1.5 billion garments are sewn by an estimated 40 million people working in 250,000 factories across countries designated by the UN as the world's least developed.

On this, the prime retail weekend of the year, it seems fatuous to say that almost nothing can come between a retailer and its rabidly enthusiastic public. Except, that is, for UK Uncut, the dynamic grassroots protest group which, during the pre-Christmas rush, succeeded in closing the flagship branch of Topshop and other Arcadia enterprises.

I felt a pang of jealousy. It's not that I don't support the cause: the highlighting of Sir Philip Green (rich as Croesus and the government's adviser on austerity) and his nifty tax arrangements through Lady Green's Monaco account is a noble cause. But I wondered why we don't get that sort of protester turnout and action when garment workers producing for Arcadia and other high street retailers are discovered to be subsisting on slave wages.

Ask, as I often do, the retailers to tell you how they have managed to make fashion unprecedentedly cheap (in the past decade clothing prices have undergone extraordinary deflation) and they will give you the usual guff about their logistics genius, deft bargaining skills and how they don't waste money on advertising and flowers in reception. This is the strategy apparently behind aviator jackets, skinny jeans and knitted cardigans that cost the same as a sandwich and a coffee.

Funnily enough, the one thing they never mention is the workforce responsible for manufacturing the giant amount of fashion consumed annually. In Bangladesh alone more than 1.5 million pairs of jeans are sewn every year by an unseen, unacknowledged army of an estimated 40 million people. This army is the engine for the Cut Make Trim (CMT)part of fashion (the point in the fashion chain where the garment is assembled and sewn) and toils in about 250,000 garment factories, predominantly in the least developed countries.

Life in the CMT army is grim, particularly in Bangladesh. A report last year by the International Trade Union Confederation gave workers there the inauspicious title of "most poorly paid in the world" and reckoned exploitation to be on the rise. But even against that unpromising introduction, the past seven days have been bad indeed.

Last Sunday riots broke out in garment factories in Chittagong and Ashulia, north of Dhaka. Or as the Bangladeshi paper the Daily Star put it, "RMG workers go berserk", featuring photographs of upturned sewing machines and work tables. Protests by garment workers aren't uncommon, but last Sunday they were met by the Rapid Action Battalion (a sort of hybrid between the police and army) and at least three protesters appear to have been shot dead.

For workers in such a risky situation to jeopardise their only source of income by protesting raises the question: how desperate are they? The answer is very. Labour rights groups have long warned that civil unrest from garment workers will only increase as food prices rise and ever decreasing wages mean many workers are attempting to feed families on less than £1 a day while working six to seven days a week. Studies show that female workers in particular find it hard to consume enough calories to sustain them through their working hours.

As we were reminded only two days later, the job of a garment worker is also very dangerous. Last Tuesday a fire broke out at the That's It Sportswear factory, also in Ashulia. The facility is run by one of Bangladesh's biggest garment export companies, Ha-Meem, and produces for global retailers including Gap. As always – there are so many fatal garment factory fires in Bangladesh that I have lost count – this was an accident waiting to happen. But this time it has received international press attention because of the size of the fatality list (still unconfirmed, it seems as if 100 people died) and its chronological proximity to last Sunday's riots.

We know that rudimentary, makeshift garment factories and fires go together. "There are three main issues that, ironically, were discussed at a conference in Dhaka two months ago," said Professor Doug Miller of the University of Northumbria, who is a specialist in ethical fashion.

"There's the electrical safety and propensity for electrical faults in these factories. There's the issues that where gangways exist they're not very clear or marked, and the whole situation can be exacerbated by the fact that garment factories can be on the 10th and 11th floors of factories. And then finally you have bad buying practices. Buyers place huge orders with short deadlines, which leads to excessive overtime and draconian rules in order to ensure orders are finished and dispatched to western clients on time. In many cases, workers are simply locked in, and that includes fire escapes."

Anna McMullen, of the campaign group Labour Behind the Label, said she was not surprised by the latest tragedy. And she said the audits offered up by big retailers as evidence of their unctuous supply chain management were virtually useless. "Inspections in these factories are often announced, and even prepared for," she said. "So a tick-list approach will always fail to check health and safety in any meaningful way. For example, on a day when there's going to be an audit, the stairwells wouldn't have been locked."

Eyewitness accounts from last week's disaster suggest one of the main gates was locked. I'm afraid this tallies with the issues I've seen myself. When I managed to get into a Dhaka garment factory last year, in a building presumably intended as an office block, producing for global retailers, I found every fire escape blocked by the 500,000-piece inventory of cut-price jeans destined for Europe.

It is too late to do anything for the 100 people killed last week making fashion for the rapacious global market, but not to rescue their families from penury now that the main breadwinner is gone. As has become tradition after this type of tragedy, a spokesman for Gap confirmed that the Ha-Meem factory supplied clothes to its stores and said the company was "terribly saddened".

"Our immediate priority is that the workers and their families receive the medical and emergency assistance they need," the Gap spokesman said. But Javier Chercoles, a former global director of corporate social responsibility for Inditex (the owner of the high street chain Zara), said it was high time international buyers and companies took responsibility for tragedies like this. Because just as global brands have mitigation strategies when they are implicated in the use of sweated labour, they often now have a strategy for when garment workers die and make the news.

They might, for example, team up with a global charity and implement a scheme or fund a children's project in the area as a way of giving something back. "That's what you do if you want to supply some balloons and toys for some children," said Chercoles, "but that has nothing to do with real rights or compensation." He pointed to a recent incident where an international retailer divided only £12,000 compensation between 65 widows of garment workers.

Instead, Chercoles has come up with an actuarial method to calculate compensation and a workable means of getting payments to vulnerable widows and widowers in some of the poorest areas on earth.

"I've applied this after two labour accidents in Bangladesh and it's been fully approved by all stakeholders," he said. But international buyers remain quiet. "Now is the time for them to step up and sign up to proper compensation rights," he said, "now that yet again we have 100 dead workers on our conscience."

It is high time, indeed. Next year is the centenary of the 1911 Triangle Shirtwaist factory disaster in New York. In that incident 146 young female garment workers were killed. It remains one of the city's biggest industrial disasters and marked the birth of the labour rights movement. The victims are commemorated in a museum and many books. But in Bangladesh there are so many garment fires that they barely register.

Our big high street names are able to shirk responsibility from the moment they place orders, euphemistically referring to a freight on board price (fob) which covers everything about the garment leaving the factory: fabric, trim, packaging and the manufacturing cost.

Very rarely is the labour cost (sometimes called the "make element") quoted as a separate item. Meanwhile, international buyers negotiate aggressively on this fob price with factories in some of the world's poorest countries. An estimated 60% of this is usually accounted for by the fabric.

Realistically, for the factory owners the only thing left to squeeze is the wage of the garment worker and they are hardly going to invest money in proper factories with functioning fire escapes and sprinkler systems unless failure to do so precludes them from getting orders. Inevitably, as buyers from our high street stores drive down the price, the slack is picked up by the most vulnerable in the chain – the garment workers.

Reporters at the Ha-Meem fire made a rare attempt to put names to faces. "The bodies identified are Farid, Maruf, Ekram Ali, Halima Begum, Maria Sultana, Ruhul Amin, Anjan, Tutul, Himel and Halima. Identity of the rest could not be known immediately," said a local report, confirming most of the victims to be young women.

This is a tragic exception. On the whole, it is still remarkably easy for retailers and consumers simply to put garment workers out of our minds, dead or alive.