It is a playful and memorable image: a charismatic actor, Gillian Anderson, splashed above the iconic logo of 007. But I was surprised by how many voices – male and female – were raised against the idea behind it when the picture was shared on social media last month. Why should there be such resistance to the prospect of a female Bond? It seems unlikely that this would be the case in any other genre. Spy fiction has proved remarkably hardy over the years, from the subtle dramas of Joseph Conrad and Graham Greene to the brash adventures of Ian Fleming or Len Deighton, and on screen with surefooted realisations from Alfred Hitchcock to Sydney Pollack. The genre may have blossomed in the fertile ground of the second world war and the cold war, but as the success of the latest John le Carré adaptations proves, there are no signs of it dying out any time soon. In fact, new writers are revitalising its themes all the time, from Ian McEwan and Helen Dunmore to William Boyd and Charles Cumming. Surely such an inventive and adaptive tradition can accommodate a woman in the lead?

Facebook Twitter Pinterest Gillian Anderson in the mocked-up poster.

Despite its richness, I have often felt alienated by spy fiction because it has often seemed so rigidly masculine, and nowhere more so than in the escapades of the evergreen Bond. Reading or watching spy narratives can feel claustrophobic when it means entering a world in which it is so often men who see and women who are seen – and seen as sexualised bodies above all.

So James Bond, peering through a huge periscope into a Russian council chamber in Fleming’s From Russia with Love, sees a girl, soon to become his lover, walk into the room, with her “proud breasts and the insolently lilting behind”. So Jonathan Pine in Le Carré’s The Night Manager, peering into Richard Roper’s inner circle as he walks into a hotel room, sees Jed Marshall, soon to become his lover, with “a view of one perfect breast, its slightly erect nipple lifted to him by the action of her arms”. So Jason Bourne in Robert Ludlum’s The Bourne Identity, about to carry out his usual nonchalant crime, stands in wait for his target’s mistress, “a windswept blonde, her large breasts stretching the blue silk of her blouse, her long legs tanned”. Perhaps that is why I was so captivated by the titles of the new film of Le Carré’s Our Kind of Traitor, which opens with the glistening body of a male dancer turning in midair; sensual, stunning and watched by women. It seemed a direct reversal of those classic Bond title sequences in which women’s bodies, side-lit or silhouetted, dance across the screen. Here, I wondered, would we be given a very different take on the sexual dynamic of the spy film?

The answer, however, was … no, not really. The film quickly moves back into more familiar territory; a world seen, above all, from the point of view of the male hero, Ewan McGregor’s Perry, a university lecturer who must learn to fight, to shoot and to kill. At one point he is given key advice by his new friend and mentor in the Russian mafia about his wife: “She’s a good woman. Keep her.” His wife, meanwhile, played by Naomie Harris, gradually turns from a clever lawyer with flashes of rebellious independence into the kind of woman so often idealised by writers of traditional spy fiction, one whose character is subsumed into the needs and desires of the male hero. These women, full to the brim with unconditional love, have decorated so many spy stories of old.

Spies and lovers Read more

This can be the case even where the hero is far from the James Bond type: Greene’s Wormold, in Our Man of Havana, is a middle-aged vacuum cleaner salesman when the lovely young Beatrice falls desperately for him and persuades him to reciprocate: “I can’t love you as a one-way traffic.” Alec Leamas, in The Spy Who Came in from the Cold, is apparently a middle-aged, heavy-drinking librarian, when the enthusiastic young Liz Gold falls for him: “I said he could come back any time. I’d wait for him always.” This pattern continues: the hero of The Bourne Identity is an unpredictably violent man with no memory and no past, yet Marie St Jacques, a successful economist, declares: “We love each other, we’ve found each other!” The very fantasy of becoming a spy is, perhaps, entwined with the fantasy of finding this insistently enthralled partner.

Facebook Twitter Pinterest Naomie Harris and Ewan McGregor as Gail and Perry in Our Kind of Traitor (2016). Photograph: Moviestore/Rex/Shutterstock

This is not to say that the women you meet in traditional spy fiction are simply decorative. On the contrary, time and again they are startlingly competent: not just great secretaries and brilliant cooks, but also good drivers and accurate shots. Beatrice Severn in Our Man in Havana is a lot more able than the book’s hero when it comes to dealing with the day-to-day business of espionage. St Jacques can be as quick as Bourne in decoding the crazy world in which they find themselves. In Deighton’s The Ipcress File, the hero first sees his lover Jean Tonnesen only in terms of the “beautiful spy” – already, in 1962, clearly a cliche – but soon appreciates the way she can capture information with her perfect memory. The woman’s useful contribution is all the more incredible because so often she does not really understand what she is doing for the hero. Denied access to full information, she acts purely out of disinterested loyalty and love.

Women make better spies - as long as they forget sex Read more

The dominance of the masculine journey isn’t necessarily a problem for female readers and viewers. The director Maren Ade, whose film Toni Erdmann was a strong contender for this year’s Palme d’Or, said recently to an interviewer: “As a woman I’m used to identifying with male characters. When I watch a James Bond movie, I’m not just the Bond Girl, I’m James Bond too.” Ade’s point of view is persuasive: the ability to find oneself looking through the eyes of others is part of the magic of fiction. If we can be a wizard, a rabbit or a ghost as we read or watch – surely any of us can be a male spy.

Certainly where the work compels us, this transformation happens: the viewer is pulled into the mind of the spy, whoever he or she is. A Stasi agent eavesdropping on a playwright in the film, The Lives of Others? No problem. An opium-addicted journalist in 1950s Indochina in Greene’s The Quiet American? Of course. For me, that shift happens most easily where spy fiction dramatises not just the certainties but also the vulnerabilities of its heroes, opening them up not only to the fantasy of domination but also the understanding of failure.

But more interesting is the growing strand of spy fiction no longer dominated by a masculine viewpoint – stories in which women are not just going along for the ride, but taking the steering wheel for themselves. Its roots go way back. The Mata Hari story may have been translated into a cliche about a loose woman who will give up everything for true love, but when Greta Garbo was chosen to take on the role in 1931 she made sure the journey was hers alone, acting all the men off the screen. Here is a spy who has already laid the ground that a Jane Bond could walk on, as she moves with unimpressed hauteur through secrets and lovers.

Facebook Twitter Pinterest Joan Crawford and Fred MacMurray in Above Suspicion (1943). Photograph: Moviestore/Rex/Shutterstock

When Helen MacInnes started writing spy novels in 1941, with a caper called Above Suspicion, she created a fresh heroine in Frances Myles, a bright and practical woman who works hard with her husband to carry out their mission in prewar Germany. I don’t want to overstate either the originality or the feminism of MacInnes’s vision – Frances ends up being captured and rescued by the men around her – but there is something intriguing in what she is trying to do. Here is a female spy with a sure intelligence, and who isn’t sexually available. When Joan Crawford took on the part for the film in 1943 she added a trademark swagger to the role.

Today, the most memorable Le Carré adaptation in a long time, the BBC’s The Night Manager, has been directed by a woman, Susanne Bier, who brought a rather different vision. Yes, there is the traditional relationship between Jonathan and Jed, in which we are very aware of his gaze resting on her as she moves through initial scenes, before she becomes his invaluable helper. But by forging a new female character, the heavily pregnant Angela Burr, out of the book’s Leonard Burr, and bringing her to life with Olivia Colman’s matter-of-fact performance, this adaptation changed the dynamic of the narrative. No longer was this a masculine cabal in which the woman was expected to assist without true understanding – on the contrary, Burr was often the only one who could see the bigger picture.

Even with Judi Dench’s M and Olivia Colman’s Burr there is a sense they’re not allowed to participate at full throttle

Facebook Twitter Pinterest Judi Dench as M in Skyfall (2012). Photograph: Moviestore/Rex/Shutterstock

And she is not alone. Once Stella Rimington had taken the top job in espionage by becoming director general of MI5 in 1992, spy fiction had to play catch up. Judi Dench’s M in the Bond films of the late 90s and early 2000s gave us a masterclass in understated power. Even if her screen time was limited, Dench gently recalibrated some of Bond’s sexist tropes with her easy assumption of authority. Still, with both Dench’s M and Colman’s Burr there is abiding sense that the women are not quite allowed to participate at full throttle – the boys have the more interesting journey in the field, while the women stand back, in a more peaceful but duller anteroom.

Female spies are 'bloody good' – and it's partly because of sexism Read more

There have undoubtedly been books that seek to change that; from McEwan’s Sweet Tooth to Boyd’s Restless, Sebastian Faulks’s Charlotte Gray and Dunmore’s Exposure, it has become much more common now, when you enter the world of espionage, to find yourself walking next to a woman. Much of the energy in Restless lies in Boyd’s ability to show that the traditional narrative of honey traps, false identities and crossed borders still shines when seen from a woman’s viewpoint. And while Dunmore’s innocent wife in Exposure almost slips into the helpmeet stereotype of old, a prim Jewish teacher from north London’s Muswell Hill, gradually Lily Callington builds her own life and her own narrative, forced into subterfuge and violence in order to survive. Her narrative is all the stronger for beginning in such a suburban and understated way.

These explorations are often historical, but new films and TV programmes that deal with the contemporary world are increasingly putting women centre stage, too. In Homeland’s Carrie Mathison we see a CIA agent as driven and intelligent as Colman’s Burr, and very much in the field of danger. The show’s writers have said that they spoke to a female CIA agent as part of their research, but, she didn’t talk “to any of us that much about gender”. And so they tried to create a person who was a spy, rather than a woman who was a spy. Kathryn Bigelow’s film Zero Dark Thirty continues that journey. “I’m not that girl, that fucks. It’s unbecoming,” Jessica Chastain’s Maya states at one point. “I’m the motherfucker that found this place,” she says at another. The film may be about the hunt for Bin Laden, but it becomes her hunt – her desires, her abilities and her actions drive the viewer towards the denouement.

Having said that, Homeland and Zero Dark Thirty are also deeply conservative works whose narratives provide a conventional defence of American power. The America that Carrie and Maya come from is fragile; it can only be protected by hypervigilance against outsiders, in particular Muslims, and Maya’s journey is partly about her accommodation with the use of torture to achieve this. So, while they are refreshing in their ability to propel women to the centre of the stage, they are predictable in their representation of political power.

In Homeland’s Carrie Mathison we see a driven, intelligent CIA agent who is very much in the field of danger

Facebook Twitter Pinterest Claire Danes as Carrie Mathison in Homeland. Photograph: Showtime

As the winners of the world wars and the cold war, and leaders of the war on terror, the world may look pretty black and white to many British and American writers and film-makers now. Certainly in much spy fiction, the world positions itself rather easily into the good side and the bad. The resulting stories are satisfying, but not always entirely convincing as a portrayal of our complicated reality. But perhaps we live in a time when reality is not quite what is wanted. No wonder then that the easy heroics of James Bond remain compelling, and when The Night Manager was adapted recently it was given a more definite closure than the book’s ending, which made it far more optimistic in its view of the ability of the good Englishman – and woman – to take on global corruption.

***

Facebook Twitter Pinterest Melinda Maclean in August, 1951. Photograph: Express Newspapers/Getty Images

My own journey into the world of spies started with a historical story, not a fictional one. It all began with a scene that I read in a biography, which stuck like a burr in my mind. That scene played out in Geneva in 1953 where an American woman, Melinda Maclean, was living a quiet life with her mother and children. In a scandal that had rocked the establishment, her British diplomat husband, Donald, had disappeared two years previously with Guy Burgess, one of the five members of the Cambridge spy ring. Fevered speculation about whether they had been spies for Stalin, for how long, what they had known and whom they had betrayed was still running through British and American society. One Friday in September, Melinda told her mother she was off to stay with friends, dressed carefully in a plain grey skirt and an electric-blue Schiaparelli coat, and drove to Lausanne station where she and the children got on a train and also disappeared.

I was mesmerised by the way Melinda had behaved, not just on that extraordinary day, but throughout her marriage to Maclean. She kept up appearances with such aplomb. While he often drank to excess, carried his secrets heavily and even broke down publicly, nobody ever suspected that Melinda knew a thing. She was the good wife, reading her fashion magazines and looking after the children. It was only much later that it transpired she too had been a devoted communist.

Her shadowy and unexplained contradictions enthralled me. When I read more about her, it was clear that she benefited from the stereotypes of the time about women and espionage. Though there had been many women working as spies during and after the world wars, they were often dismissed as the seductive courtesan (Mata Hari) or the martyred nurse (Edith Cavell), and neither of those stereotypes seemed relevant to Melinda. In fact, there had always been clever and committed women around the Cambridge spies: Donald’s first handler was a woman, a Soviet agent called Kitty Harris, and Kim Philby was brought into espionage by another woman, an Austrian communist called Litzi Friedmann. When Melinda thought about her likely fate, she would have had before her the horrifying example of the Rosenbergs: Julius and Ethel had been executed for passing secrets to the Soviet Union that very summer of 1953. Nevertheless, Melinda somehow managed to steer clear of any suspicion, and after Donald’s defection, she was assumed to be so entirely innocent she was not even interrogated.

I longed to tell Melinda’s story, yet as I travelled along its twists and turns, her reality seemed to resist my explorations. Gradually, I found myself letting go of the real Macleans and turning to invention instead. I fell in love with the heroine I created, with her apparent foolishness and her hidden cleverness, her idealism and her gradual disillusionment. Although there was no evidence that Melinda was ever a spy, my fictional Laura becomes an active spy. I felt that in bringing her into this secret work I could give fuller expression to her desires and their consequences.

I also felt that her actions as a spy gave me a way to magnify what all women go through; a strong sense of the importance of appearances, which must be kept up regardless of what lies beneath. I fell so deeply into her particular story, it is only now that the novel is finished that I can look with fresh eyes at the great tradition of spy fiction – and I am struck by its richness and complexity, developed by so many great writers and film-makers over the years.

Facebook Twitter Pinterest Illustration: Dwayne Bell

For me, there are two abiding strengths in the genre. In those traditional capers written by John Buchan and Fleming, there is a fantasy about the possibility of individual agency, of living a life in which your own desires – for sex, for violence, for anarchic behaviour – are realised, along with almost superhuman abilities to evade capture or death. While others go about their lives in ignorance, only you can see further and escape the traps that others lay for you.

But in those darker and grittier tales, such as those told by Greene or Le Carré, the very fact of being a spy can make escape impossible. You are not only watching, you are also being watched. In these narratives, everyone’s latent sense of paranoia is made real, and far from being able to escape the traps laid for him or her, so often the spy’s own actions become the final trap. When you enter a book such as Le Carré’s The Spy Who Came in from the Cold or a film such as Francis Ford Coppola’s The Conversation (1974), you find yourself entering a spy’s life where the dream of true escape or of full knowledge is shown to be just that, a dream. And the fears and attractions of the spy’s life are there for women as well as for men; there is nothing particularly masculine or feminine about paranoia and suspicion. As Rimington once wrote: “I have heard it said that women make particularly good intelligence officers, both spies and counterspies … because they are discreet … psychologically tough, and better than men at keeping their own counsel. I think all that is pretty much nonsense … you need people of varied qualities and talents, and you find them among both men and women.”

I realise now that I wanted my heroine to share the same kind of spy’s life that many fictional spies have, male and female. I wanted her to have dreams of control and escape, but as she lives and learns, I wanted her to recognise the awful inescapability of the path that she and her husband had chosen, and how one can never get away from the consequences of one’s actions. Because spy fiction gives one a way to explore such universal realities, it seems to be a tradition that constantly reinvents itself, that never stays still and that will remain alive, as long as we have secrets to keep and fears about the secrets of others.

• A Quiet Life by Natasha Walter (Harper Collins) is published on 16 June.