When it comes to the best of anything, what do you expect? If it’s science fiction and fantasy novels you want epic adventures and getting out of impossible situations. But what you often get is barely disguised sexism and inability to imagine any world where women are involved in the derring-do.

At the end of 2013, after a year of reading very little, I decided to embark on a challenge: read all the books I hadn’t yet read on NPR’s list of 100 best sci-fi and fantasy novels. Nostalgia permeates the list. Of the books I read, there were more books published before 1960 than after 2000. The vast majority were published in the 1970s and 1980s. There were also many sci-fi masterworks or what were groundbreaking novels. However, groundbreaking 30, 40, 50 or 100 years ago can now seem horribly out of date and shockingly offensive.

This is seen in The Forever War, one of the rare books that portrays homosexuality. The main story centres around military recruits, sent to fight newly discovered aliens but in relativistic time-scales. As they spend most of their time travelling at relativistic speeds, time on earth slips hundreds of years into the future. However, despite the same years in combat, one of the supporting female protagonists, Marygay, only gets made an executive officer while Joe, the male protagonist becomes a major.

What was more disquieting was the military regulations about enforced promiscuity of the women soldiers:

The orgy that night was amusing, but it was like trying to sleep in the middle of a raucous beach party. The only area big enough to sleep all of us was the dining hall; they draped a few bedsheets here and there for privacy, then unleashed Stargate's eighteen sex-starved men on our women, compliant and promiscuous by military custom (and law). . .”

Maybe it was assumed that the men would be promiscuous but this wasn’t mentioned. Given that both the men and women were soldiers, and equally powerful as they were all enhanced by exoskeletal combat suits, why would only the women have to be compelled to sleep with men by military regulation?

What I hated, and dreaded the most as I continued to read through the list, was the continued and pervasive sexism - even in seemingly progressive books for their time. I devoured science fiction and fantasy when I was younger - the idea that I was also devouring patriarchal and sexist ideas made me deeply uncomfortable. The fact that these were all supposed to be the best of the genre, was even more shocking.

Testing sexism in sci-fi

In anger, after I read the first 10 books or so, I made my version of the Bechdel test, adapted for books. I thought I could ask for a bit more than films because there is more time for exposition and exploring complex ideas.

The test had three simple questions:

1: Does it have at least two female characters?

2: Is one of them a main character?

3: Do they have an interesting profession/level of skill like male characters?

It was staggering how many didn’t pass. Some failed on point 1, like Sword of Shannara. In a vaguely Lord of the Rings-inspired universe, a young man goes on a quest to save the world, all the while being chased by various evil orcs and minions. After mentions of three dead mothers (of the male lead characters), there’s finally a woman about three quarters in: a seemingly intelligent and educated woman named Shirl. However, after the man she rescues (after he rescues her) awakes, he says to her: "Don't ask questions now, just do what I say."

You wonder why Shirl had to be there at all. In this book and others, it felt like the book would have been less sexist if there weren’t any women at all. At least that way, they wouldn’t be belittled or be treated contemptuously.

For the rest of the book, Shirl merely hangs onto her hero’s arm and has no further impact upon the plot. She is the only woman in 664 pages and is only there to be an object of desire for two male characters.

Many failed on my second criteria, like Out of the Silent Planet or Rendezvous with Rama.

C S Lewis' Out of the Silent Planet was one of the oldest books on the list, aside from Jules Verne. It’s an early attempt at explaining space flight and encountering an alien race. Most of the plot revolves around the main character, Ransom, trying to understand the aliens before managing to escape back to earth.The most entertaining aspect of the book is the ludicrous physics. There is one woman in the story, who Ransom exchanges about three sentences with before she wanders off. Perhaps you can forgive that on age, the book being from 1938.

The same can’t be said for Rendezvous with Rama, which was written in 1973. It was critically acclaimed and won many of the main science fiction prizes such as the Nebula Award, the British Science Fiction Association Award, the Hugo Award and Locus Award. The story centres around a group of space explorers who have to investigate a mysterious spacecraft that enters the solar system.

While there are more women, almost all are subordinate to the main male lead. There is one female authority figure who is on the Council of Rama (the organisation directing the efforts of investigation), but she doesn’t play a significant role. I also got distracted by the fact that, inexplicably, the male lead sleeps with almost all the women mentioned in the book.

Finally, most would fail on the third part of the test because the women characters were all mothers, nurses or love interests. They were passive characters with little agency or character development, like the women in A Canticle for Leibowitz and Magician. They were scenery, adding a tiny bit of texture to mainly male dominated world.

A Canticle for Leibowitz is set in a post-apocalyptic future, where monks now study to recover "lost science". Of course, being set in a monastery of sorts, there are few women, none of them remotely integral to the plot.

Magician, at 841 pages, was sure to have women, I thought. The premise contains some promise, there’s a rift between two worlds and an army from one world invades the other. Peace is only achieved when the main character obtains powers of wizardry and manages to force the end of the war through magic.

So there are women, but I don’t think any needed to exist for plot. At one point I had some hope that the elf queen would be a check on the solely male dominated leaders of the book. However, my hopes were quickly dashed when she becomes smitten with one of the male leads. Any potential capability she has is quickly supplanted by her consort’s growing otherworldly power and influence over the elves. She basically spends most of the time worrying about him. Most of the other women solely existed to be fancied by the various men in the book and to fill some of the 841 pages.

From bad to worse

From the other side of it, fantasy novels all seemed to be black mirror distortions of the medieval period. The first one of these, Dragonflight, was a caricature of the medieval period with unrealistic levels oppression and lords being able to sexually exploit whatever lower class women they wanted.

The plot revolves around a blight from space (the threads) that will wipe out everything on that planet unless the mostly male heroes manage to defeat it (of course). For some reason, the best way to burn these threads is with dragons.

The main male and female characters hate each other, but their dragons love each other and so whenever their dragons mate, well, you get the picture. However, there’s a darker, more violating aspect to this: “He had been a considerate and gentle bedmate ever since, but, unless Ramoth and Mnementh [the dragons] were involved, he might as well call it rape.” Yet, this doesn’t seem to stop him.

For some reason, the book wants the reader to cheer on a rapist (and this isn’t the only book that asks you to do this). These sort of characters made me hate a book. I couldn’t like a character that raped someone. I couldn’t care less if the world was destroyed. How could anyone? Is this really the best the fantasy genre has to offer?

The absolute worst offender was Final Empire, which was like an unpleasant caricature of Dragonflight. The world was supposed to be gritty - but to an unrealistic and ultimately unbelievable extent. Normal people had no rights, women were killed by the nobles if they slept with them (to prevent half-breeds), ash literally rained from the sky and the nobility had super powers with the ultimate bad guy seemingly immortal.

Women having no agency is merely an additional layer of dirt to the already soiled world. And it’s all so casually mentioned: “'You know the law says that a lord can bed any skaa woman that he wishes?' Vin Nodded. 'He just has to kill her when she's done.'”

It was the repeated emphasis of the relative powerlessness of women, their status of objects or things to be won, that almost makes me want to write off the whole genre as a lost cause.

Judging from the books on the list, some fantasy novels had a depressingly limited lexicon when it came to describing non-obliging female characters: bitch, whore, slut. Reading one or two over a year and you might not see the problem but after about 30, you start to look at the genre in disgust.

There was also the utter lack of imagination when it came to putting women in danger. While male characters were faced with death (which, is a pretty good motivating force for anyone) there were so many books where to put a woman in danger was to have her raped, threatened with rape or threatened some other sort of sexual servitude.

After reading quite a few books in a row where rape or attempted rape was used solely to add an element of danger, the only conclusion you could have was that it was bad and lazy writing.

There were also no real consequences of the rape or sexual assault when it did happen. One example of this is in Lord Foul’s Bane, one of the most miserable books on the list. The main story is about a leper, who is transported to another world (which he doesn’t believe is real) and is no longer a leper. The people in this world think he’s there to save it from the oncoming evil.

Thomas Covenant, the main character, actually rapes a young woman and is astonishingly unrepentant for most of the book. Quite late in the story, he has one brief moment of realisation of what he did and then he moves on, with no character growth or resolution. He’s an absolutely horrible character that we’re supposed to like or want to continue reading about. Staggeringly, this was on the first book in a long series.

Even where a main female character is raped, they just kind of get on with life. Like in The Diamond Age, where Nell knows she’s going to get raped: “She supposed it was inevitable that, in due time, these men would take those liberties with her that they have ever been claimed as angary by irregular fighting men, who have willfully severed themselves from the softening feminine influence of civilized society, with those women who have had the misfortune to become their captives.”

Of course, this does happen: “On two of these occasions, Nell was outraged in the manner she had long suspected was inevitable.”

But then, she just puts it out of her mind (if it even had an impact) and makes her escape. Being raped twice seemed to have no effect on her whatsoever.

There is never any engagement with what people would think, feel or do in those situations. Like Final Empire, it feels like it’s been used as a lazy shorthand to add some edginess into the world. Despite something so invasive and violating, it’s never something that affects a character’s development (which you figure it would). It’s the equivalent of just having a bad day.

There were also books that were outright misogynistic, like a A Spell for Chameleon where characters openly talk about not trusting women: “'Women are the curse of mankind,' Crombie said vehemently. 'They trap men into marriage, the way this tangle tree traps prey, and they torment them the rest of their lives.'”

The main plot of A Spell for Chameleon is that the main character, stupidly named Bink, has no magical talent and unless he gets one, he will be kicked out of his magical kingdom. Along the way, he meets Chameleon, who has the unenviable magic of being smart but ugly in one phase of the moon and beautiful but stupid in another. This inevitably leads to Bink liking her:

She was growing lovelier by the hour. Her personality was not changing much, except as her diminishing intelligence caused her to be less complex, less suspicious. He liked that personality - and now, he had to admit, he liked her beauty, too. She was of Xanth, she was magic, she did not try to manipulate him for her private purposes - she was his type of girl.”

Apparently for Bink, having someone compliant was more valuable than intelligence or independence, making Bink an utter creep.

How does this make it on the list of 100 best science fiction and fantasy books? Again, I point you towards nostalgia.

Recent controversy

I’ve been acutely aware of the ferocious debate in the science fiction and fantasy community about representation (the recent controversy over the Hugo awards is a case in point), so I should have been somewhat prepared for this relentlessly depressing string of disappointing novels. But with the dogged determination of reading through the list, it really grinds you down. How can people actually defend the status quo when it’s so obviously awful? Putting these books on a pedestal, given how problematic the portrayal of women and other minorities, is surely part of the problem?

Frankly, from my vantage in 2015, it was just plain weird to read books where there were hardly any women, no people of colour, no LGBT people. It seemed wholly unbelievable. I know what you could say: it’s science fiction and fantasy, believability isn’t one of the main criteria for such books. But it is relatively absurd that in the future people could discover faster-than-light travel, build massive empires and create artificial intelligences but somehow not crack gender equality or the space-faring glass ceiling.

The consequence of the lack of women and the obvious sexism is that the books became very much like one another. My book reviews contained more profanity and I became a much more harsh critic of the genres I most enjoyed reading. They were all the same story of white guys, going on an adventure.

Some diamonds in the rough

In contrast to the male-dominated stories, there’s The Doomsday Book, where a woman named Kivrin is put into all sorts of danger. She’s stuck in 14th century England, with her meticulously crafted cover blown by illness, and only her knowledge, strength and intelligence to help her survive.

The story shifts between Kivrin’s time in the future, where everything is going wrong in the attempts to get her back, and the 14th century where Kivrin has been sent. I adored the conceit of having time travel manipulated only by historians, which is the reason Kivrin is sent back in time in the first place.

However, the danger she gets put into is all existential, for example the inability to travel back to the future, the plague and the suspicions of the women in the household. She doesn’t have to worry about being attacked or raped, but rather not to give up more of her already broken cover.

It makes for a much more suspenseful and richer storytelling experience. Not surprisingly, it was one of the best books on the list. It felt like there was actual jeopardy and that each action had a consequence.

Another one of my favourite books, A Fire Upon the Deep, had some equal-opportunity jeopardy: the complete annihilation of the universe. The plot is extremely bizarre involving a self-replicating entity devouring the universe, different physics of space travel depending on the proximity to the galactic core, a world where the dominant intelligence are pack minds and two children making first contact and accidentally triggering a civil war.

Two of the main characters are women. One is in a spaceship rushing through the weird physics of the universe, being pursued by the self-replicating organism. The other a teenage girl on the planet with the pack minds, trying to find her brother and survive the civil war. One of the main pack minds is also predominantly female (given than they were made up of more than one mind, they weren’t all male or female).

What made these books compelling and interesting were their plots, but also the ability for me to get totally lost in the story. Some of the other books would be jarring in their sexism or just lack of women - their world building failed.

I can understand how many of the books on the list may have once been groundbreaking but that doesn’t mean that they are now the best examples of the genre. They have been supplanted, hundreds of times over, by other authors that took similar themes but made them better and more inclusive.

Fighting my own nostalgia

Having spent many hours reading terribly sexist fiction, swearing and coming close to throwing books across the room, I wonder how many of my favourite books would fare with a re-read. Would they pass the test? I have many warm fuzzy memories of The Foundation Series but after reading The Caves of Steel I hesitate to re-read it. Asimov’s characterisations of women have less emotion than his robots, and his writing was less compelling than I remember. I might leave them to nostalgia - but if I ever get asked what the greatest science fiction or fantasy novel is, it will be something I read as an adult.

I think A Fire Upon the Deep and Doomsday Book deserve to be on that list, but few others.

If I could suggest one book to add to the list (as it came out after the list was compiled) it would be Ancillary Justice by Ann Leckie.

Ancillary Justice is the story of a sentient spaceship, seeking revenge for the murder of its lieutenant. It has the most elegant literary trick of having only having one gendered pronoun, which is female. It was amazing to have the normal male-gendered universe being completely reversed. You can sometimes tell what gender a character is when the author chooses to describe them in a way that makes it obvious but most of the time, you don’t know.

After reading so many of these books, you can readily see that there is a problem with science fiction and fantasy novels when it comes to representation of women and minorities. What those people defending the lack of women or minorities are doing is advocating for a genre to remain stagnant. They are defending nostalgia. They’re trying to ingrain a conservative strain in a genre that was radical when it started. They’re arguing on the side of repetition, terrible storytelling and awful characterisation.

We deserve better as fans of the genre. We deserve to be challenged by radical visions of the future like those who first picked up Jules Verne. We also deserve to see ourselves reflected in the characters so we in turn can be inspired to help create that future.