The US supreme court has rejected the biggest sex discrimination case in history, ruling that the claim against retail giant Walmart on behalf of as many as 1.6 million women was too big to bring to trial.

All the top US judges ruled that the 10-year-old gender bias case failed to meet the standard for class action cases, and a majority of conservative judges also ruled that the women did not have enough in common to pool their claims.

The judgment is a major victory for Walmart, which had faced billions of dollars in claims if it had lost.

The women are free to bring cases on an individual basis but legal experts said they may be unwilling to do so. The decision was also described as a blow to all future class action law suits against companies or anyone trying to use the courts to push for systemic change.

The suit was originally filed in 2001 on behalf of Walmart employee Betty Dukes and five of her co-workers, who claimed they had been passed over for promotions and were paid less than male co-workers.

Their lawyers had provided statistical evidence that women earned less and were promoted less often. They alleged that Walmart's corporate culture and employment policies fostered gender stereotyping across the company and that Walmart knew of discrimination but failed to act.

The retailer's lawyers argued that the firm had no case to answer and had a stated anti-discrimination policy.

A lower US court ruled that the case could go to trial, but Walmart appealed to the supreme court. If the class action had been accepted it could have set a precedent for gender discrimination at many corporations, with companies such as Microsoft and General Electric writing to the court expressing their concern.

Led by conservative supreme court justice Antonin Scalia, the court found several problems with the law suit. All the judges agreed that it failed to meet a technical requirement for its type of class action primarily concerned with monetary claims.

Scalia and the conservative judges on the supreme court went further and argued the women did not have enough in common to represent a class of people. Scalia sided with Walmart's argument that there would need to be common elements tying together "literally millions of employment decisions at once" to make the basis for a class action case. He said that such a common element was "entirely absent here."

Legal expert Stuart Slotnick of New York law firm Buchanan Ingersoll and Rooney said the ruling "changes everything in Walmart's favour".

He said large-scale class actions would now be far harder to bring against other companies as greater proof of system-wide discrimination would be needed. Often such proof is only available after a case has been granted class action status and the process of discovery – where lawyers can demand access to sensitive internal documents – begins.

"Walmart was facing tremendous pressure from a case where so many claims were being made against it in one case before one judge," said Slotnick. "Now each individual will have to find a lawyer to fight their case and I would question whether most individuals will want to do that," he added.

Melissa Hart, a constitutional law expert at the University of Colorado school of law, said the US courts were increasingly "hostile" to cases that seemed to be pushing for systemic change.

"It's another signal from the courts that they are not going to let you push for change through the court. The courts are saying they are about individual harm, not systemic change," she said. "In the 1970s and 1980s there was a lot of hope that a law suit could change the world. This court is saying: 'No, it can't'."