British television changed in 1991, when DCI Jane Tennison (steadying herself outside the door, taking a deep breath, fixing a cool expression on to her face) walked into an incident room filled with a sneering, jeering, sniggering, lewd, matey, loyal band of detectives who were almost all male: a rugby team of lads, incredulous that someone in a skirt was to take charge of a murder investigation, humiliated by having a woman boss. The drama of who killed and mutilated the female victims ran alongside the drama of a woman battling in a man’s world: how could Tennison withstand the hostility and outright bullying of her colleagues and bosses, and at the same time manage her private life? She had to be tougher than the men at work and still soft and tender at home, placating her lover, apologising to him, cooking for him, compartmentalising her world, though of course the boundaries kept crumbling and collapsing. In the lonely spaces in between, she stood in corridors, visibly collecting herself for the next fight; she smoked ravenously. She was her own battleground.

Produced by a woman (Sally Head), written by a woman (Lynda La Plante) and starring a woman (Helen Mirren), Prime Suspect turned the familiar detective show inside out, dismantling the world that had become so familiar on TV, where maverick male detectives were the experts and women usually the victims – the abandoned body, the mutilated object on the floor, legs splayed and throat cut and dead eyes staring up at us, the clue that needed solving. It was an exhilarating spectacle of female assertiveness and protest, and of its bitter personal cost.

Twenty-three years later, the lonely figure of Jane Tennison has been joined by a thickening crowd of other women; an exception has become a trend. Female detective dramas have almost become their own genre. Move over Poirot, Wexford, Morse, Frost, Bergerac et al – for many of whom time barely seems to pass, and whom experience does not scar – to make way for Gillian Anderson’s DSI Stella Gibson (The Fall, which is returning next month), Olivia Colman’s DS Ellie Miller, (Broadchurch, the second series of which is scheduled for 2015), Lesley Sharp and Suranne Jones’s DCs Janet Scott and Rachel Bailey (Scott & Bailey, the fourth series of which started last month), Vicky McClure’s DC Kate Fleming (Line of Duty), Sarah Lancashire’s Sgt Catherine Cawood (Happy Valley), Brenda Blethyn’s DCI Vera Stanhope (Vera). And let’s not forget Sofie Gråbøl’s Sarah Lund (The Killing) and Sofia Helin’s Saga Norén (The Bridge). Women are solving crimes now; women are exploring our terrors, doubts and anxieties for us. And very terrific and odd women they are.

Brilliance: The Bridge’s Saga Norén, played by Sofia Helin, is among the new wave of genre-redefining female TV detectives. Photograph: ZDF/BBC

For this female cast often bring their own psychodramas into the traditional whodunnit, making it rich and bleak and murkily complicated. They are themselves mysteries; they resist easy solutions and the dynamic momentum of plot, which drives forward in spite of the repeated tugs of red herrings, and gets tangled up in the downward pull of character, in the labyrinths of memory, sadness, anger and guilt. Fictional detectives are often loners, but being women makes them doubly alone. Many thrillers are about good and evil, but these thrillers are about being human, flawed and in trouble. They make us care not only about the outcome – the satisfying narrative click is still there, if sometimes a bit muffled – but also about the characters. We identify with them, fear for them, want them to be happy, know they won’t be, want to own their shirts, or jumpers, or coats. For a while they are more real than our reality.

The Killing, which was in the front line of the new female-led detective series, had a plot that was addictive and yet creaked with inconsistencies. It was assembled from hefty building blocks of misdirection. But flowing around these, washing through every crack in the investigation, was the intimate stuff of ordinary life: the slow and terrifying unfolding of grief, the aftermath of horror, the grubby and impressionistic portrait of a city, streets half-seen through car windows, where the rain falls and light doesn’t come and the fog shrouds buildings in strangeness. And at the heart of this was Lund, little and pale and stern, and most wonderfully grumpy. Wearing that jumper that spawned a thousand copies, chewing that gum, not speaking when expected, making mistakes and never apologising, letting down her boyfriend, letting down her son, behaving terribly, not smiling, not explaining, not agreeing, not listening, not being womanly. Not a good girl at all, but an intractable, unstoppable force.

Gender changes meaning. If Sarah Lund had been Sean Lund, her behaviour wouldn’t be particularly remarkable or taboo-breaking. Not being there for her son, arriving at family occasions late or not at all, being curt: that’s what men with important jobs do all the time. It is easier for them to break the rules, since they made them in the first place; indeed, the rule-breaking, the violence and the hard drinking seem part of what makes them effective detectives. Women’s behaviour, by contrast, is judged against the norm of their male colleagues: it can never be invisible, never taken for granted. And for a woman to behave as a man often does sets up a conflict in the viewer as well – we want her to be like this, but we also don’t because she’s swinging a wrecking ball through her life. Some of the most nerve-racking moments of the series involved not the tracking of the murderer but the moments when Lund’s jaw clenched and we knew she was about to do something that she might not regret but that we partly would. Her demented pluckiness radicalised the plot.

Demented pluckiness … Sofie Gråbøl as Sarah Lund in The Killing. Photograph: Tine Harden/DR presse

If The Bridge’s Norén had been played by a man, everything would have changed: the moment when she walks up to a stranger in a bar, for instance, asking if he wants sex, would not give us the same frisson of discomfort and delight. A male would not have set us alight as Norén did with her social blindness, her brilliance, her role as truth-teller and, in the end, as the conscience of a drama that investigates the murky world of crime and exposes fault lines in society and in the self.

Happy Valley’s Sergeant Cawood is doubly an outsider, because Cawood is not just a female police officer but a grandmother – not so young any more, or glamorous, but bashed about by life and now on a journey that will take her back into her own past. This is a series written by a woman, Sally Wainwright, that – through one extraordinary ordinary woman – can examine decades of damage in a family and a community. While it has a dynamic story, it also bores down through the strata of guilt and love and grief and failure. Happy Valley is superbly made and beautifully acted, especially by Lancashire, whose face is etched with a life of sorrow and endurance, and whose character is so encumbered by baggage that the series almost resembles a high-quality misery memoir in uniform, or a female northern gothic (the music in the opening credits is very like the music from the southern gothic detective series Justified). Cawood is the sister of a heroin addict; her daughter was raped, had a child by the rapist, committed suicide, and this in turn broke up Cawood’s marriage. Her ex-husband has remarried but they still sleep together. And this is before episode one has even begun. She has so much on her mind, no wonder she forgets to call for backup when going down into a dark cellar alone. These female detective dramas are a very Protestant genre: people carry burdens they will never shake off; character is an accretion of memory and guilt.

When a male detective spits in the hair of DC Fleming in Line of Duty, it matters that a man is spitting at a woman. It makes it perverse as well as ugly. When DCI Gibson, in The Fall, examines the body of a sexually abused and murdered woman, it makes all the difference that a woman is looking at a woman – that a living woman is touching a dead woman’s body, staring at the wounds, imagining what took place. A woman is hunting a serial rapist of women, and there is an intimacy between the worlds of the living and the dead, a connection.

Fetishised: Gillian Anderson’s portrayal of DSI Stella Gibson in The Fall is subject to some lingering camera work. Photograph: Steffan Hill/BBC/Artists Studio

The Fall explores notions of femaleness and sexual violence and it does so in a way that is powerfully unsettling and sometimes queasy-making. The camera lingers on its central character: her strongly beautiful profile and the full curve of her lips; her sleek hair, her gorgeous silk shirts (almost as iconic as Lund’s jumper), her shapely calves, the way she looks as she swims, as she undresses. She is itemised, fetishised, turned into a body, watched and assessed. It can feel that the way the serial killer watches his victims is eerily replicated by the way the camera watches Gibson. She complicates this by her own sexual behaviour; aloof, icy, sexually passionate without being warm, she uses men the way that men traditionally use women. She turns them into objects, the way that women are turned into objects by the male gaze or, at the other end of the spectrum, by the rapist.

Gibson, like Tennison or Lund, destabilises the traditional whodunnit. Fictional male detectives in the past have often been robust figures of competence, standing at the centre of the plot, from where they make sense of the incomprehensible, turn chaos into order, join up the clues to find the criminal, restore normality. But we no longer have such a belief in authority (the “Evening, all” of Dixon of Dock Green), in disinterested genius or in absolute answers. The world we live in now is more tentative, contingent and compromised; the doctor, the priest and the detective can’t solve everything. The lover won’t come like a knight on a charger to rescue the woman in distress (in fact, it’s better to beware the lover). We have only ourselves to depend on. We are our own redeemers because there is no God, though there is still Freud, and the notion of Manichean good and evil has been replaced by things that are murkier and less comforting. The Fall, or Broadchurch, or Happy Valley or Line of Duty are not neatly resolved; lives have been wrecked and grief cannot be assuaged. It uses the old tropes to make new meanings. There can’t be happy endings any more. Female detectives represent this new kind of reality because they often become implicated in the stories they are trying to make sense of. Women, however defended they are and strong, have a vulnerability about them simply because of their gender.

This porousness of boundaries is at the heart of Broadchurch. Colman’s heart-wrenchingly touching DS Miller seems at first a more traditional female character than her fictional colleagues. For a start, she isn’t in charge but subordinate to David Tennant’s Alec Hardy. Hardy is the brooding, silent, complicated one with the tragic backstory, while Miller seems to have a life of domestic stability, almost a stereotype were it not for the poignancy Colman brings to the role. Miller is happily married and has a son; her manner is practical and motherly. She puts warming mugs of tea into Hardy’s thin, cold hands, comforts people, responds with instinctive kindness to the sorrow of others. But (spoiler alert) it turns out that the horror they are trying to hunt down is inside her own home, her bed, her heart; she’s been lying night after night beside a paedophile and murderer.

Heart-wrenching … Olivia Colman brings poignancy to the role of DS Ellie Miller in Broadchurch. Photograph: Patrick Redmond

For in this female world, the detective is also a victim. The walls between the professional and private worlds collapse and this allows the viewer to identify with the character, as we can never identify with the expert, the invulnerable or the flawless. Few of TV’s female detectives become enduring staples in the way of Morse, Wexford and the rest – perhaps because the pressure of the women’s interior worlds must always explode outwards. They cannot be the stable centre of a drama lasting years or decades.

Perhaps Scott & Bailey will prove the exception to this rule of loneliness and instability: a complicated and intimate female friendship and working partnership lies at the heart of the show (which was created by women and written, again, by Wainwright) and this friendship is the foundation for its success and staying power. There might be frictions and rivalries, but the two detectives share secrets and a wry humour, drink pints of beer and glasses of wine together, bring humanity and wit to a world of poverty and gruesome murder. The two of them and their female boss normalise female authority in a way that a woman alone cannot.

Detective novels recently have been full of unreliable narrators. Gone Girl and Before I Go to Sleep are two of the most interesting examples of the linear form of a whodunnit being derailed by the narrative voice; there have been thrillers told by characters suffering from Alzheimer’s disease, desperately trying to keep the pieces of their world in place and making a coherent picture out of fragments. The slipperiness of memory and the self-deceptions of the mind compromise the notion of an absolute truth. (I write psychological thrillers with my husband, Sean French, under the name of Nicci French: when we chose to have a female psychotherapist, Frieda Klein, as the protagonist of our series, it was because we felt a detective of the mind could more satisfyingly explore contemporary anxieties and because a woman is always in some sense an outsider, who does not and cannot belong to the old world order.) In the same way, many of the new female-detective dramas challenge the familiar realism of the genre – and of course realism is a style like any other, that lays a template over the mess of life and gives the fictional illusion of order and completion. Female detectives tend to bring instability into a story because they are always on the margins, having to negotiate in a man’s world and refreshing a complacent genre with a new self-awareness.

Intimate: Lesley Sharp and Suranne Jones as Janet Scott and Rachel Bailey in the acclaimed series Scott & Bailey. Photograph: Tony Ward

I bought The Fall as a box set (the phrase “binge-watching” has just been added to the Oxford Dictionary), having read reviews that were almost unanimous in their acclaim. But I couldn’t hurtle through it. Halfway into the first savage episode I started to cover my eyes, looking through my fingers and then not looking at all. Finally, I had to turn it off and for many months couldn’t return to it, because I had been so unnerved and horrified by the level of cruelty towards women. The serial rapist and killer watches his chosen victim, follows her, toys with her, tortures and obliterates her; we do not see her as a subject in her own world, but as an object – the object he has chosen. These slow, drawn-out scenes are intercut with scenes in which Gibson has sex. And just as the rapist toys with and tortures his victim, so the camera toys with the viewer, giving agonising moments of hope before the final extinction.

The Fall powerfully explores sexual violence and the way in which serial rapists and killers eroticise power and death, but there’s a very fine line between exploring violence and male misogyny and simply portraying, even enacting it. I couldn’t work out if it was feminist or almost pornographic in its visceral depictions of degradation and sexual horror. Perhaps it is both – and perhaps that’s why it is so powerfully disturbing. But I wonder if the series could have got away with its portrayal of the sexual torture of women if it hadn’t had a strong professional woman at its centre. Did Anderson’s DCI Gibson legitimise the portrayal of sexual horror?

Alfred Hitchcock famously said that thrillers were about making women suffer. In a recent piece in the New Statesman, the actor Doon Mackichan passionately attacks mainstream TV drama and film for feeding the culture that sees violence against women as entertainment. She writes that she will no longer act in any drama with a storyline involving “violence against women”, unless it has a radical feminist agenda. She is partly echoing what the thriller writer and reviewer Jessica Mann wrote in her now famous diatribe against sadistic misogyny in contemporary crime fiction, in which “young women are imprisoned, bound, gagged, strung up or tied down, raped, sliced, burned, blinded, beaten, eaten, starved, suffocated, stabbed, boiled or buried alive”. And she adds that female writers are as guilty as their male colleagues.

I’m writing as one of those women increasingly troubled by the violence in our genre. It’s a fine line, a grey area, a slippery slope. In Happy Valley, we see a young woman kidnapped, brutalised, sexually assaulted and drugged in a series of extended sequences across six episodes. We are immersed in a world of suffering. Mackichan wants dramas that do not involve violence against women. But the world is full of misogynist violence and art will always be drawn to areas of darkness and trouble. Look at fairytales: even little children need a safe way to explore horror and cruelty. Women do suffer and women are raped, and while it’s a fine line to tread between what is justified and what is gratuitous, at least now there are a great many brilliant, strong, determined, heroic women detectives in fiction – if not yet in fact – who can help them. Women saving women. Women saving themselves.