Vivienne Charles has a problem: the heroine of Lilah Pace’s erotic romance Asking for It (2015) can experience sexual satisfaction only if she imagines herself being raped, which causes her endless shame. Enter the rugged, mysterious Jonah Marks, who ‘looks as though someone took the macho ideal of a masculine form and brought it almost to the breaking point’.

Jonah instantly arouses Vivienne’s wildest fantasies, filling her mind with visions she abhors yet which excite her tremendously, ‘visions of him bending me over the back of my car, pushing up the skirt of my sundress. Of him pulling me into the backseat, putting my hand on his cock, whispering, Time to thank me.’ When Jonah finds out about these fantasies, he offers to fulfill them; like a true gentleman, he wants to ‘rape’ her in a way that will be safe and comfortable. But will Vivienne be able to overcome her own self-loathing to enjoy the consummation of her darkest desires?

The true hero of Asking for It is unmistakably Doreen, Vivienne’s intelligent, sympathetic psychologist. ‘Rape fantasies are among the most common sexual fantasies women have,’ she tells Vivienne (and the reader). ‘The fantasy isn’t your problem; it’s the extremity of your fixation on it.’

With Doreen’s careful guidance, Asking for It takes the reader on a journey through the gap between fantasy and reality, celebrating the deep arousal lingering at the borders of consent. These are ideas that implicitly inform many books in the romance genre. While most aren’t as explicit as Asking for It, the trope of the domineering, protective and well-endowed (physically and financially) alpha male is still a staple, as are power differentials between the hero and heroine. And though it’s not as common any more, questionable consent – even rape – haunts the genre’s recent past.

Evidence of the enduring hold of these tropes on the female imagination can be found in the runaway success of E L James’ Fifty Shades of Grey (2011), about virginal Anastasia, and Christian, the billionaire who wishes to own her. Selling more than 125 million copies worldwide, the series remains hugely popular despite being panned for its writing. It also contains scenes of explicit sadomasochism.

The Fifty Shades phenomenon manifested an uncomfortable paradox, one that gets only starker with every gain of the women’s movement. Why do women who have never been so free, so empowered, and so highly aware of the importance of consent continue to enjoy these retrograde tropes?

‘I am of the opinion that a genre that is written by women, for women, about women, about the female experience, even if that experience is codified and structured within patriarchal, established boundaries, is inherently feminist,’ says Sarah Wendell, co-founder of Smart Bitches, Trashy Books, a US-based website devoted to reviewing romance novels through a critical lens.

Wendell grants readers of all socioeconomic and educational backgrounds permission to enjoy their fantasies – be they feminist, sadomasochistic, paranormal, or male-on-male. For Wendell, this permission is the genre’s beating heart. ‘With romance, you are placing a centrepiece, a focus, on women’s sexuality as a healthy and important thing,’ she said. ‘Her orgasm is important! And so is her security, and so is her ability to access birth control.’

Unlike other spheres where female consumers are ignored, here women are everything. These readers populate a unique subculture. Devoted and voracious fans, they are catered to by an industry that recognises their value and by authors grateful for their interest. Unlike other genres, which treat women as accessories or plot devices to motivate a male hero, here women are the plot.

The very contradiction at the heart of romance fiction is a lesson: within feminism lies the permission, even the imperative, to enjoy, even if the fantasies you enjoy are not very feminist.

Romance novels have long been derided by the literary establishment as bodice-rippers manufactured to engage the lurid fantasies of frustrated housewives. Often, their authors suffer public disdain, viewed as the sordid peddlers of a mysterious and unfortunate contraband – female desire.

But many books that are cornerstones of literary history are essentially romances. Take what’s considered one of the first novels: Pamela, Or Virtue Rewarded (1740) by Samuel Richardson. It follows the melodrama of a 15-year-old maid who resists the aggressive sexual advances of her master, only to be rewarded with a marriage proposal. Pamela possesses all the hallmarks of a romance – a heroine you can root for, an alpha male to fall in love with, and a happy ending, or H E A as it’s known in the industry, for ‘happily ever after’. Today, Pamela is regarded as a classic of literature, but at the time it received its fair share of mockery, including a spoof, Henry Fielding’s Shamela (1741).

Throughout the 18th century, the ‘literary’ novel was shaped largely in opposition to romances by authors such as Eliza Haywood. Haywood was riotously popular. She wrote about women of great sexual appetite, who go so far as to masquerade as others for a single night in bed with the men they desire. But like today’s romances, Haywood’s work was regarded by the literary establishment as salacious, unrealistic and feminine. As William Warner, a professor of 18th-century studies at the University of California, Santa Barbara, has argued, in order to gain legitimacy, ‘real’ novels – such as Daniel Defoe’s Robinson Crusoe (1719) and Fielding’s Tom Jones (1749) – coded themselves as moralist, realist and – perhaps most crucially – masculine. Their authors gained credibility by creating false dichotomies between themselves and their female counterparts. It was only in the 1990s that feminist scholars began to chip away at the mythology that Haywood’s work was so different from Defoe’s or even Fielding’s, resurrecting authors such as Haywood and revealing their importance in the history of the novel.

The modern romance novel is usually dated to 1972 with the publication of Kathleen Woodiwiss’s The Flame and the Flower, a Victorian-era story featuring Heather, a penniless virgin who is mistaken for a prostitute and raped by Brandon, a landowner and captain of a ship. When Brandon is forced to marry the now-pregnant Heather, they set sail from London for the US, and the long voyage to their H E A begins.

‘What would it take to make you feel safe?’ Jonah asks Vivienne when they sit down to plan her ‘rape’

In Beyond Heaving Bosoms: The Smart Bitches’ Guide to Romance Novels (2009), co-authored with Candy Tan, Wendell explains that romances from the 1970s and ’80s tended to feature a brutal hero who more often than not would rape the very young, virginal heroine, through whose exclusive point of view the book is represented.

But in the early 1990s, all that started to change. Romances quickly began to evolve into their current form – books that feature more emotionally complex heroes whose point of view is also represented, as well as what Wendell calls ‘the kick-ass sexually experienced heroine’. Rape was replaced by ‘forced seduction’, where the heroine’s love for the hero stems from an initial encounter in which consent is ambiguous rather than absent. These days, even forced seduction is out. Consent – and contraception – are now par for the course in most romances, with societal norms infiltrating the plots to reflect what readers want to read. In romances such as Asking for It, where power play and non-consent are themes, significant portions of the narrative are devoted to negotiations between the partners so the play will be mutually enjoyable. ‘What would it take to make you feel safe?’ is the first question Jonah asks Vivienne in Asking for It when they sit down to plan her ‘rape’.

Today’s romances have lots of variety, too. There are subgenres – historical, contemporary, inspirational, young-adult, romantic-suspense, and paranormal-romance. They feature, among other things, older women, rich women, women CEOs, women who are clan leaders, women who are political leaders, and more often than not, full-figured women. While women of colour still struggle with issues of marginalisation in the overwhelmingly white industry, the past four years have seen exponentially more writers – and more characters – of colour, due in no small part to the opportunities for self-publishing offered by the internet.

In 2013, Americans spent $1.08 billion dollars on romance novels, which represented a whopping 13 per cent of the adult-fiction revenue – double what literary fiction brought in the same year. And unlike many other forms of entertainment, romance sales were undisturbed by the economic downturn of 2008, a year in which reportedly one in five Americans read a romance novel.

From an industry perspective, this slice of the pie has not gone unnoticed. ‘Every publisher wants a piece of that readership,’ says Cindy Hwang, vice president and executive editor at Berkley Books, an imprint of Penguin. ‘There’s a realisation that romance readers are voracious readers who are always looking for new books. It’s a very engaged readership that will always spread the word and talk about new books that they love, and it’s absolutely a readership to be respected because they have a lot of influence on what booksellers sell.’

To truly understand the power of the romance industry, there’s one place you have to go: the Romantic Times Booklovers Convention (RT), a week-long extravaganza for romance readers featuring workshops, costume parties, books, swag, and signings with authors. This year, RT was held at the Hyatt Regency in Dallas, Texas, and the variety of women present matched the genre’s heroines: lawyers, cops, engineers, mothers, teachers, booksellers, waitresses and city employees walked around in groups of three and four, chatting excitedly about cherished books, beloved characters, favourite authors.

RT is a universe unto itself. Its 3,000 attendees represent a subculture free from the male gaze. Mostly in their 40s and 50s, these women sported naturally coloured hair and comfortable-looking clothes and shoes. It was a place where not only are older women not ‘invisible’, as countless Daily Mail surveys suggest, but they are valued and respected, and a few are even millionaires.

Ana Markovic, 31, was waiting to meet her favourite writers, Charlaine Harris and Kresley Cole, both Number 1 New York Times bestselling authors of fabulously popular urban fantasies and paranormal romances. She had travelled from Iceland to attend RT. Originally from Serbia, she works as a draftsperson, and in her spare time consumes more than 150 romance novels a year.

‘A certain percent are making a ton of money, and a lot of other women are making basically a living wage’

Ana is unwavering in her devotion to romances. ‘I like to swoon,’ she explained. But it’s not really about the sex scenes. She’s in it for the tension, the drama, and most importantly, the ending. Other genres have tension and drama, but only romance guarantees the H E A – an implicit promise that the payoff will be worth it, that the pain will transform into pleasure. When an author once broke the implied contract of romance and killed off a main character, there was such reader outrage that, in the next volume, the character was brought back from the dead.

The romance industry is one in which women carry clout, and lots of women can become financially independent, explained Mary Bly (pen name Eloisa James), a New York Times bestselling author, over breakfast at RT. ‘A certain percent are making a ton of money, and a lot of other women are making basically a living wage,’ she said. And although it’s a competitive market, it’s also a very nurturing one: Bly said she needs to help as many writers as she can, because their books sustain her readers while she writes the next Eloisa James novel.

Bly writes historical romances that tend to feature smart, resourceful heroines who steal the hearts and minds of powerful, alpha royals. In Four Nights with the Duke (2015), Mia has a secret, successful career as a romance novelist, but for altruistic reasons involving a favourite nephew, she must get married. She blackmails Vander, a local duke, into marrying her, albeit planning to dissolve their marriage after six months. Not if Vander can help it, for he has grown attached and, now that they are married, must possess her, body and soul. In A Kiss at Midnight (2010), Bly’s retelling of the Cinderella tale, Kate has been forced by a greedy stepmother to masquerade as her stepsister to please a prince. Bly’s books use historical constraints, not scenes of degradation, to highlight the heroine’s struggle for equality. They crackle with wit, almost like a raunchy Jane Austen, and Bly manages to marry historical plausibility to contemporary – and very sexy – pacing and dialogue.

Somewhat ironically, today’s historical romances are a bastion of progressive stories. As the historical romance writer Heidi Bond (pen name Courtney Milan) put it to me: ‘I don’t think I have fantasies that stem from disempowerment. I think I have fantasies that stem from empowerment. But in many ways, that empowerment only has meaning if you can show its lack, and so I think I write about overcoming disempowerment.’

‘The dominatrix is a huge facet of male desire, but we only call it “anti-feminist” when women have it’

Other books are less straightforwardly progressive, and the old tropes persist. Cole’s popular novels in particular play with questions of consent. In her book The Master (2015), the fast-talking, sensuous Catarina becomes an escort out of sheer desperation. Her first client is mob boss Maksimilian, who tends to favour tall, slim, blonde Russian escorts, not short, curvy Latina ones like Cat. But the sex between Maxim and Cat is so explosive that he can’t resist her, nor she him, even when he imprisons her in his penthouse. In another, The Warlord Wants Forever (2006), the beautiful Valkyrie Myst is hunted down by Wroth, the vampire who can’t forget her. Wroth manages to break Myst’s supernatural chain, forcing her to obey his commands – some of which involve spontaneous orgasms. The scenes of dubious and questionable consent often pit the heroine’s own desire against her agreement to the sexual encounter.

What to make of such books? Asking for It author Lilah Pace – whose name is a pseudonym for a best-selling young-adult novelist – resists the idea that desire is somehow political. ‘We always have moral panic when it comes to female sexuality,’ she told me. ‘God forbid a woman should know what she wants sexually and it should be different from what a guy wants to do.’ There’s power in asking for your fantasies, no matter what they are. ‘I’ve been really disturbed since Fifty Shades came out and everyone was saying: “Is this what women want? To be treated badly?” as though no one ever heard of a dominatrix. That’s a huge facet of male desire, but we only call it “anti-feminist” when women have it.’

Bly is the rare author who allows that a story about securing the love of an alpha male is indeed less than feminist. She sees it as a reflection of a biological, rather than a psychological, imperative. ‘You’ve gotta accept physiology at some level,’ she said. At some very basic, biological level, women are looking for someone who can protect their progeny.

The evolutionary explanation for the female attraction to alpha males extends to the question of rape fantasies, which between 31 and 57 per cent of women report having. In The New York Times Magazine article ‘What Do Women Want?’ (2009), Marta Meana, a professor of psychology at the University of Nevada, told the journalist Daniel Bergner that women’s desires are not relational, but rather narcissistic. Bergner reported that to understand Meana’s scientific approach, one should envision a scene of a woman pinned against an alley wall being ‘ravished… Here, in Meana’s vision, was an emblem of female heat,’ Bergner writes. ‘The ravisher is so overcome by a craving focused on this particular woman that he cannot contain himself; he transgresses societal codes in order to seize her, and she, feeling herself to be the unique object of his desire, is electrified by her own reactive charge and surrenders.’

But this scene, in some ways so evocative of romance fiction, is also off in crucial ways. For one thing, today’s heroines are a lusty lot. Her desire doesn’t stem from the hero’s attraction but her own. Rather than a man ravishing a woman in a dark alleyway, the archetypal scene of today’s romances would be a man and a woman negotiating over the parameters in which they will enjoy each other’s bodies, a negotiation that occurs especially if the sex depicted is going to involve ravishing.

What romance novels teach us is that the answer to the question of why women enjoy fantasies involving an alpha male and power differentials must be a narrative one, not a physiological or evolutionary one. The problem, after all, is a narrative one. As Doreen says in Asking for It: ‘The fantasy comes from… a culture that eroticises violence against women, and leftover puritanical guilt about sex that tells us we’re not allowed to choose it and want it for ourselves and from God only knows where else.’

The sociologist Eva Illouz at the Hebrew University of Jerusalem has a theory. In her book Hard-Core Romance: Fifty Shades of Grey, Best-Sellers and Society (2014), she portrays the fantasy of female disempowerment as a plot point in the larger script of romantic love. Women don’t fantasise about disempowerment for its own sake so much as they fantasise about the kind of love that is predicated on their disempowerment, in which the rituals of love are more straightforwardly in the hands of men. Like Courtney Milan, Illouz believes that what women enjoy is the spectre of empowerment that’s promised to them if they engage in courtship rituals that give power to men. And historically, they are justified in believing this. Traditionally, women were entirely disenfranchised in all areas – socially, legally, financially. There was one domain where they were adored and worshipped, at least at the level of cultural representation – the domain of love.

‘It was the only place where women would hope to be worshipped and in a way escape their general social condition,’ Illouz told me. The fundamental script of traditional romantic love, found in most poetry and literature, is of men adoring women, reversing the usual disempowerment of women, even compensating for it. In the Western world, until the 1970s, this was romantic love: men adored women, and at the same time deprived them of social and economic power. ‘That’s why I think love has been historically so damn important for women,’ Illouz said. ‘Because it was the only moment, the only experience where they were important.’

Romances resolve the constant tug-of-war in modern life by reconciling feminism with surrender

Fast-forward to consumer culture and the women’s movement, and this model collapses. Women are still far from achieving true equality, and they still do more of the housework and childrearing. This duality, according to Illouz, has them hearkening back for the kinds of adoring love that compensated for their inequality – the kind of adoration they find in romances.

This is also how Illouz makes sense of the popularity of Fifty Shades of Grey. ‘The equality that has been promoted by 40 years of feminism demands ongoing, ceaseless negotiation,’ she wrote in a 2012 article for Der Spiegel. But Fifty Shades stops the endless bargaining by ‘setting up and freezing caricatured and exaggerated roles and positions’. The narrative makes ‘inequality acceptable because it is consensual, contractual and pleasurable’.

Romance might touch so many because it resolves the 21st-century tug-of-war between feminism and surrender. After all, if the modern romance heroine is desired by a dominant man, she also sets the parameters by which she fulfils her own desires. At the crossroads of the conflict and the key to working it through is the issue of consent.

In a workshop at RT called ‘Walking the Line of Consent’, the author Jeffe Kennedy reviewed the continuum from non-consent (‘Non-Con’) to questionable consent, or even dubious consent (‘Dub-Con’) – situations where the heroine agrees to sex, and even enjoys it, but has been trapped or pressured into it. This is what happens in Kennedy’s novel Under Contract (2015), where a financially destitute woman agrees to have kinky sex in exchange for paying off her debt. ‘Not a real thing at all, right?’ Kennedy said, laughing. ‘At least, nobody’s ever paid me.’

Of course, in real life, consent is everything, the gold and only standard. But in fiction, the most important variable is the consent of the reader. ‘You’ve all probably read those books where the heroine is all: “No, no, no!” and the hero is all: “Oh, but you will! I’ll force you!” and you as the reader are going: “Yes you will!” That’s the reader giving consent,’ Kennedy explained. Part of why dubious consent or forced seduction works is because readers also identify with the hero. ‘When the pirate captain takes the shrinking English virgin, we’re doing that too,’ she said. ‘We want to consume her. We want to seduce somebody. We like being on both sides of that, at least as far as the fantasy.’

As readers, we are drawn to what Kennedy called ‘the dark lure of being overpowered’ – by an alpha male as well as by a good story. Rather than exposing something about female desire, compromised consent in romance novels exposes something about narrative: conflict is what makes a great story. ‘Consent in fiction is psychological dynamite. It’s about subconscious desires and sexual fantasies. It’s not about real life,’ Kennedy emphasised.

As if to represent that dark lure of the flesh, RT also had a handful of male models vying for spots on the covers of new books. In a gender inversion of a high-rolling party, they strutted around in shirts with the sleeves cut off, flexing sculpted muscles and flashing brilliant smiles, hoping to catch the eye of publicists and authors. No one paid them much mind, and they seemed symbolic of the tension in romance novels – the centrality of an alpha male in a genre for, by, and about women.