So it's been a bad week. On Monday, I had to switch off Classic FM when it played Wagner, since he was an antisemite. Then I burned my copy of The Hunger Games, because it's all about children killing one another for entertainment. Finally, I threw my television out of the window because Breaking Bad's Walter White is a drug-dealing psychopath, and the show's fans love him for it.

OK, I didn't do any of that, because that would be absurd. If we rejected every creative work that is in some way "problematic", the canonical cupboard would be bare. So why are so many people I meet so shocked that, as a woman and, whisper it, a feminist, I enjoy playing Grand Theft Auto?

The GTA series certainly isn't female-friendly. Loved by teenagers, it has become one itself: yet in the 16 years of its existence, it hasn't once offered players the chance to experience its sprawling scenery and high-octane thrills as a female character. In the latest instalment, released last week, women are once again pushed to the margins.

You can rescue one whose car has broken down, spy on another having sex with her boyfriend while she checks her phone, and, of course, visit the obligatory strip club and "make it rain" bank notes. There aren't any female characters to root for, be impressed by, or even fall in love with. It was the same in the last game: women were there to nag you, or be bribed – whether with fancy dinners or cold, hard cash – into having sex with you. Women are GTA's wallpaper.

Given all this – and the way the series has allowed you to visit a prostitute, then kill her to get your money back – perhaps the horrified reactions shouldn't be surprising. When I went on Channel 4 News to talk about GTA, Jon Snow asked: "Aren't you worried, as a woman, that the whole thing isn't a bit creepy?"

Well, yes, it is creepy, but worried isn't the right word. I'm bored, more than anything, as well as irritated that another generation of young players isn't being offered something more exciting than this. "The medium has grown up, and now the GTA franchise is a giant juggernaut that appears to be punching down instead of up," says female games journalist Leigh Alexander. "I think that's why its problematic elements rankle – not because I'm 'offended', but because it seems lazy, repetitious. I'm not 'offended' that I can't play as a woman; I'm disappointed at the missed opportunity."

That feeling isn't unique to video games. Playing the new Grand Theft Auto, I realised I feel about it the way I feel about Martin Amis novels. I admire the cold brilliance, the execution, the imagination: but I will never love that world, and I never feel inclined to return to it once the story is over. In both London Fields and GTA's Los Santos there's no place for me; I can visit, but I'll never feel welcome.

What distinguishes games from books, or films, is that the dodgy sexual politics and wanton violence of one is used as a stick to bash them all. Partly, this is down to the old hangover of the idea that games are for children, when the average age of a player is now 30. Then there's the idea that games are a "niche" or "geek" pursuit, even though GTA V took an astonishing £500m worldwide in its first day on sale. Finally, there's the spectre of violence. Games are an easy scapegoat for horrific acts committed by those who happen to play them, despite the lack of evidence of any causal link.

An unpleasant feedback loop has developed: between the sneers and the moral panics, gamers have become hyper-sensitive to criticism. We have a chip on our shoulders the exact shape of a tabloid front page. And that makes it hard to articulate the idea that while the medium is thrilling, exciting and innovative, individual titles can too often be derivative, nasty and riddled with stereotypes.

The Grand Theft Auto series is certainly guilty on the last two counts. The novelist and critic Tom Bissell has described the protagonist's Jewish lawyer in 2002's Vice City as "an anti-Semitic parody of an anti-Semitic parody", while in the new game one of the main character's daughters has a tattoo that reads "skank", and one mission involves you helping a paparazzo capture a starlet's "low-hanging muff".

That's enough to make me sigh, but not give up. My own red line in games is the unthinking use of rape, whether as a "character building" incident for female protagonists (but, strangely, not for male ones) or as an "edgy" version of a sex scene. In the run-up to GTA V's release, a disturbing forum posting went viral. "I want to have the opportunity to kidnap a woman, hostage her, put her in my basement and rape her every day, listen to her crying, watching her tears," it read.

Now, I would flat out refuse to play a game where you could do that, regardless of whether it was forced on me by the storyline, or merely an option in sandbox play. So am I being a hypocrite running people over in my virtual car without a pang of remorse? I don't think so. "In games the aim is, usually, to beat your opponent before they beat you," says Simon Parkin, who writes for the New Yorker. "It's a game of reactions, speed and quick planning based upon principles of fairness – or of the protagonist overwhelming the odds. This is in no way analogous to rape, which is all about an assertion of power in an unequal equation. If you're threatened with rape, you don't respond in kind."

For the same reason, I am uncomfortable about GTA's torture scene, where you extract information, along with a tooth, with a pair of pliers. Enjoying violence when it's not a "fair fight" feels very different to being quicker on the draw than an armed enemy.

This is where game fans' defensiveness becomes a problem. It takes a brave critic to slaughter a sacred cow – particularly by pointing out sexism in games – because the backlash can be so fierce. Anita Sarkeesian, who appealed for crowdfunding to make a web series about female characters, became the subject of an internet hate campaign as a result. On the #1reasonwhy hashtag, hundreds of women vented about why they were such rarities in the game industry: they felt their work was dismissed, their opinions were ignored, and some were subject to sexual harassment at conventions.

That is not a message that many gamers want to hear, nor do they want to confront the fact that this is a mature enough medium to face proper scrutiny. When Carolyn Petit of GameSpot called GTA V "politically muddled and profoundly misogynistic", the review attracted more than 20,000 comments, many of them viciously hostile. "GameSpot shouldn't have given GTA V to a woman to review … their input is pointless," was one of the most printable.

Yet Petit was incredibly positive overall about the title, giving it 9 out of 10, and she tells me that the GTA series is one of her all-time favourites. "I'm much more engaged by a game that makes meanings I find problematic than by a game that doesn't attempt to mean much of anything at all," she says. "If creators genuinely want to put out works that celebrate misogyny, they are free to do so, and we are free to talk about their work critically."

She's right: the conversation needs to be more sophisticated. Without decrying game violence as a symptom of our moral decline, we need to explore how that violence is presented, and what it means within the game world. The same goes for misogyny.

So yes, Grand Theft Auto is offensive – deliberately so, because controversy sells. Yes, it is "problematic" in its treatment of gender, race and sexuality. And yes, it is also an incredible technical and artistic accomplishment. In other words, it is just like any other milestone in any other creative field. Fans and detractors alike need to stop the special treatment.

Helen Lewis is deputy editor of the New Statesman