Before anyone accuses this so-called Gang of Three of sour grapes, just imagine the fuss if it was one of the clubs competing for the title that had fielded an ineligible player and escaped a points deduction. Imagine if it was Manchester City with Fernandinho, or Chelsea with Willian, or Liverpool with Mamadou Sakho, rather than Sunderland and Ji Dong-won. Then consider the way the piped music of the Premier League would have turned into a heavy metal moshpit. There would have been uproar.

Alternatively, drop down to the other end of football to understand a little more about why Norwich, Cardiff and Fulham believe Sunderland were let off too lightly in the Ji case and have instructed lawyers to take it up with the league. Look at what has happened to AFC Wimbledon over the past week, docked three points for fielding an ineligible player. Or the mother of all punishments that left Altrincham, an even smaller club, on their knees a few years ago.

At their level, Altrincham will always struggle to be heard above the din of the higher leagues. Yet it is some story. As one campaigner wrote in a letter to the Football Association at the time: "Altrincham's crime was not to cheat, nor was it to engage deliberately in any type of misconduct, it was simply to sign an English footballer from another English club in good faith."

The player in question, a Liverpudlian by the name of James Robinson, had been scouted playing for Accrington Stanley reserves. What Altrincham did not know was that he had previously had a short spell at IVB, an Icelandic club, and that meant they needed international clearance. A cross was marked in the box when it should have been a tick. Accrington were fined £250 but nothing more because he never played in their first team. Altrincham were not so lucky. The 18-point deduction is the highest doled out for this kind of offence – and it meant automatic relegation from the Conference. As Geoff Goodwin, Altrincham's chairman at the time, remembers it: "It ripped out my heart."

They were reinstated the following season because Scarborough were demoted for breaching financial rules and Canvey Island withdrew from the league. That, however, is not the point, and it is not too difficult to guess what they make of Sunderland getting off with a fine from the Premier League for not having Ji's international clearance. Goodwin's take on it, when we spoke a few days ago, was short and to the point. "The football authorities in this country look at the smaller clubs and just stand on them," he said.

It is tempting to wonder what they make of it at Oxford City, too, bearing in mind they were docked three points in March after fielding a player they did not realise should have been suspended and have just been relegated from Conference North – by two points. For them, there has been no Altrincham-style reprieve and there is already a human cost. The manager, Mike Ford, has just been informed that dropping down a division has cost him his job.

On it goes. Harrogate Town, another Conference North club run by a skeleton staff, were docked three points in September for fielding three players in a match when their registrations had not been completed properly. Wimbledon's case came about because a virus had closed down the club's offices. The member of staff who was meant to sign off the forms was ill and the message never reached the team's manager. Another genuine mistake and, again, one that had to be punished.

It is not that these clubs have lost points that is strange. It is that, higher in the leagues, another club kept all theirs. "All you want is some level of consistency," Goodwin says. "Instead, it's one rule for one and one for another. That's unfair." And he is right.

That is partly what has compelled Norwich and Cardiff to bring Fulham on board and start exploring the possibilities of legal action. They will inevitably be accused of meddling and it is easy to imagine it being thrown back at them – with a degree of justification – that if they wanted to stay in the league they should have won more games rather than relying on one of their rivals losing points through an administrative error. Equally, they are entitled to fair play and openness and there is something profoundly unsatisfactory about the way Sunderland's breach of rules B14.5 and U11 was handled. There has never been any explanation about why the relevant authorities were so keen to keep it schtum. These stories always come out sooner or later and the attempt at secrecy created the suspicion there was something to hide. Memo to Richard Scudamore: this is an industry that finds it almost impossible to keep a secret.

Sunderland cite the fact that three of the four league games in which Ji played were defeats. Yet who thought up this theory that playing badly should mean getting away with it? The club have still broken the rules over a sustained period, no matter how their games panned out. Every case is different, naturally, but Gus Poyet is correct when he says there should be a clear punishment that everyone knows. There is for falling into administration, or racism, or any manner of other offences – just not for fielding an ineligible player.

For the record, Sunderland drew 1-1 at Southampton when Ji was a second-half substitute, and they also beat Milton Keynes Dons 4-2 in the second round of the Capital One Cup. Sunderland were losing 2-0 when the South Korea international was substituted after 49 minutes but, again, that is missing the point. The Football League had the option to throw Sunderland out of the cup. Instead, a fine was issued, followed by another pact of secrecy, and Sunderland went all the way to the final, earning an awful lot more money than they were made to hand over in return.

So what happens So what happens next? My suspicion is that the Gang of Three will ultimately become a Gang of One (whoever finishes third from bottom) if Sunderland stay up. Poyet described his team as needing a "miracle" after losing 5-1 at Spurs last month but, since then, they have thrashed Cardiff, come within two minutes of winning at Manchester City and been the first-ever team to condemn José Mourinho's Chelsea to defeat at Stamford Bridge. The 1-0 win at Manchester United on Sunday leaves them three points clear of the relegation zone with two home games to follow against West Brom and Swansea. The miracle is almost complete.

But it is not just Norwich, Cardiff and Fulham who consider it unfair. Steve Bruce and Sam Allardyce, the managers of Hull and West Ham respectively, have said the same. Other clubs, such as West Brom, have expressed concerns.

A few miles from the Stadium of Light, Seaham Red Star were automatically docked three points by the Northern League this season because of a mix-up concerning the forms for a loaned player. To recap, this is what the league's chairman, Mike Amos, said of the registration process: "Sometimes it's not just a wrong 'un but a right wrong 'un – a deliberate attempt at cheating. On other occasions it's the result of an innocent and unfortunate mistake." Seaham, he said, fell into the second category, but the deduction was mandatory so it would not "leave us open to allegations of bias or worse".

It is a shame the Premier League did not have a more defined system because what we have now is getting messy. It also has the potential to become a great deal messier and, when staying in the top division is worth tens of millions of pounds every year, nobody should really be surprised it has come to this.