More than 4000 people have joined a class action against the federal government's bungled robo-debt scheme, as Centrelink overhauls the debt-recovery regime.

Peter Gordon, senior partner at Gordon Legal, filed a class action on Wednesday and joined with Bill Shorten, who is Opposition spokesman for government services, to call for every single case of "income averaging" to be refunded.

Felicity Button, 31, was unable to buy antibiotics for her one-year-old daughter after $11,500 was withdrawn from her bank account without warning in 2017.

"I just burst into tears," the Victorian nurse said.