In her teens, Linda Green idolised her local football team. But the years of domestic violence and sex scandals surrounding the game mean she can't cheer for England now

As a teenage girl, I was obsessed with football. The love affair began in 1981 when, aged 11, I decided that football was the new ponies – down came my posters of Black Beauty, up went everything Tottenham Hotspur. From 1983, I was a regular at White Hart Lane, cheering from the terraces, and, during the school holidays, the Spurs training ground. Fans could stroll in, watch the players and have photos taken with their idols. This was an era before the media obsession with Wags, when the players were married to teachers and nurses, not models and pop stars, and, to the best of my knowledge, none of them took advantage of any of the young female fans.

I'm not saying the Spurs team in the 80s were all saints, but my own idol Gary Mabbutt was, and still is, a fantastic ambassador for the game, working alongside Nelson Mandela to help bring the 2010 World Cup to South Africa.

When I moved away from London in 1989, hooliganism in football was thankfully fading, but sexism, racism and homophobia were still rampant. Big money poured in from Sky TV at the end of the 80s, and then, in 1992, came the creation of the Premier League when football became a really lucrative business and the players celebrities.

By the mid-90s, my relationship with the game was increasingly acrimonious and from the last Premier League match I attended, when Spurs were playing my local team, Coventry City, all I can remember was the barrage of homophobic abuse directed at Tottenham's David Ginola by his own fans. I knew it was time to hang up my scarf.

From then on, I watched in dismay as a breed of arrogant young men was created. Players used and abused women – and knew they were untouchable. Their clubs were too financially dependent on their skills to cut them loose.

When Euro 96 fever hit Britain, it suddenly became cool for women to be football fans. But the ugly side of football was ever more evident. I still remember the photos of the battered face of Paul Gascoigne's wife Sheryl after he smashed her head on the floor at a hotel in Scotland in October 1996. Yet the football establishment failed to act. Gazza was considered, as former England manager Bobby Robson once said, simply "daft as a brush". It wasn't until a few years later, when Gascoigne was caught committing an apparently more serious crime – eating a late-night kebab – that he was finally dropped from the team.

In 1998, it was Ulrika Jonsson's battered face in the tabloids, courtesy of former England star Stan Collymore. Another media outcry, but again the football establishment remained silent. Over the next decade came a stream of allegations of binge drinking, addiction, nightclub brawls, rape, domestic violence, serial adultery and "roasting" sex scandals. By the time I gave birth to my son Rohan in 2004, I wasn't looking forward to taking him to football matches. Thankfully, he's never asked.

Some say footballers just reflect our society, their crimes magnified because of their celebrity. To an extent, this is true. It's not necessarily surprising that young men with huge disposable incomes, often from poor educational backgrounds, find it difficult to resist temptation, or lack a strong moral code. But what is deplorable is that none of the big clubs or the Football Association has come out strongly against the worst of their behaviour.

Instead, we hear depressing accusations about footballers such as John Terry, who allegedly had an affair. He was stripped of the captaincy, but will still be playing for the team at the World Cup. Terry's replacement as England captain was Rio Ferdinand – who was previously banned for missing a routine drugs test and has clocked up four driving bans. Injury has now ruled Ferdinand out, so the captain's armband has gone to Steven Gerrard, who – with only one drink-driving conviction and having been cleared of affray in a bar brawl two years ago – is the obvious "clean-cut" candidate. Well, who else could it have gone to? Ashley Cole perhaps, if it hadn't been for the allegations of adultery and debauchery. Wayne Rooney? Maybe, as Colleen has forgiven him for soliciting a prostitute earlier in their relationship.

The truth is that I would no rather have my son parading around with the name of Terry, Ferdinand, Cole or Rooney emblazoned on his back than I would take him to the local magistrates court to cheer on those who have been found guilty of a similar array of misdemeanours. And I would urge other parents to think about the sort of role models they want for their children before shelling out for that mini England kit.

I won't be cheering on "our lads" during the World Cup. I can't. They don't represent me or my country.

Linda Green's new novel, Things I Wish I'd Known, is published by Headline Review (£6.99).