The quickest way to an audience’s heart is to kill off one or both of your character’s parents. Aladdin, Bambi, David Copperfield, Frodo Baggins, Han Solo, Inigo Montoya, Jane Eyre, Little Orphan Annie (of course), Mowgli, Pippi Longstocking, Snow White, Tarzan, Vito Corleone, Professor X: all were parentless.

Orphaned protagonists are so common that an online encyclopaedia of narrative tropes has more than 25 pages on orphan-related themes, including ‘Street Urchin’ (Oliver Twist), ‘Disappeared Dad’ (Forest Gump) and ‘Doorstop Baby’ (Harry Potter). A page on ‘Death by Childbirth’ (Luke and Leia Skywalker) underscores the trope’s appeal by pointing to its statistical improbability:

Parental Abandonment occurs with an overwhelming frequency in fiction. On top of that, an overwhelming number of victims lose their mothers during childbirth. So sad, so tragic, so heart-wrenching … such a goldmine of a plot device. Nothing impossible about it, but the statistics are ridiculously high, especially for any industrialised nations…

People in the West aren’t the only ones to tell tales about triumphant orphans. The most common element in the stories of the Igbo from Nigeria is that ‘the heroes and heroines were motherless children, orphans or paupers, who always vanquished the more privileged by miraculous or magical means’, as Ifi Amadiume writes in Male Daughters, Female Husbands (1987). Meanwhile, as Harry Ignatius Marshall writes in The Karen People of Burma (1922), ‘many tales recount episodes in which an orphan exercises his uncanny powers, usually in defence of some weaker person whom he saves or helps to get the better of his foes’.

The exalted orphan is just one of many tropes that pervade the world’s literatures. For more than a century, scholars, booklovers and screenwriters have scoured humanity’s stories in search of universals. They’ve identified all kinds of wonderful patterns, from horny world-creating tricksters to abandoned child-heroes who go on to establish nations. But perhaps the most far-reaching pattern is what I call the sympathetic plot. The sympathetic plot is so familiar, so entrenched in how we understand stories, that even folklorists passed it over. Probing the sympathetic plot can explain why people engage with stories, as well as what sets apart the stories told today in the West.

The sympathetic plot is simple. Protagonists have an understandable goal, such as marrying royalty or killing a monster. They confront obstacles, often in the form of mean opponents, eventually overcoming them through their skills or outside assistance. At the end, they receive rewards, such as power, prestige or a life lived happily forever after.

This structure – the goal, the obstacle, the rewards – makes up the core of the sympathetic plot. But at least four secondary features often pop up too. They don’t appear in all sympathetic tales, but they’re common.

First, protagonists are usually appealing. When the literary scholar Jonathan Gottschall studied characters from around the world, he found that, whenever protagonists’ looks were mentioned, they were almost always described as attractive. In fact, of 568 female protagonists coded, only eight were described as unattractive.

But there are other traits aside from good looks that make protagonists appealing. They’re often witty, brave, industrious or big-hearted. They’re capable of feats impossible for normal folk, sometimes in ways so ludicrous they border on the comical. Heroes who battled monsters, for instance, could kill villages of cannibals (as in stories told by the Mbundu of Angola), shoot arrows through armour (the Apache of the American Southwest), snipe high-flying condors at the age of four (the Tehuelche of Patagonia), and decapitate ogres with the slash of a thumbnail (Micronesia).

Protagonists adventure: they traipse through unfamiliar worlds and face off against sequences of terrible obstacles

Second, protagonists suffer early misfortunes. Like the characters listed above, they’re often orphaned, though they can also be abandoned or the children of poor rural folk. Some stories pile several of these tragedies on top of each other, as with Spider-Man, who starts out parentless but quickly loses kindly Uncle Ben too.

Another way to subject characters to immediate disadvantage is to have their older siblings bully them. Game of Thrones (2011-19) went wild with this technique, using it in relationships between the Hound and the Mountain, Daenerys and Viserys Targaryen, and Tyrion and Cersei Lannister. And, like orphaned protagonists, tormented younger siblings appear all over. For instance, the Russian anthropologist Andrei Golovnev observed that the folktales of Siberian reindeer-herders ‘usually begin with the oldest brother dominating the younger ones, but end with the youngest becoming the leader over the rest of his male siblings’.

Third, the protagonists’ opponents are mean and intimidating. They might be popular kids, sour coworkers or autocratic teachers. They might be cruel stepparents, callous stepsisters or pompous rival princes. Or they might be monstrous. They might have scarlet eyes, slits for nostrils and skin whiter than bone. They might have killed innocent children; they might wear a skull-shaped helmet over their lava-charred face. They might be giants.

All over the world, opponents are atrocious. Some have many heads, as in stories told by ancient Indians, the Nkundo of the Democratic Republic of the Congo, and the Métis of Canada; others have mangy skin, as told by the Ainu of Japan; or eyes on their knees, as told by the Guahibo of Colombia. One is cursed with testicles so large that he has difficulty walking, as told by the Apache.

Finally, protagonists adventure. They traipse through unfamiliar worlds and face off against sequences of terrible obstacles. Sometimes, the protagonists’ goals propel them on their adventure, like when they set off to slay dragons. Other times, circumstances fling them into the unknown and they need to jury-rig their way home, like when a tornado whisks Dorothy to Munchkinland. The adventure is the glue binding the narrative together: it establishes an overarching goal, such as destroying a ring or getting home, and then riddles the route with tricksy hurdles and redirection.

Scholars have recognised the sympathetic plot for as long as they’ve compared stories, but they’ve buried it in elaborate templates. In the early 20th century, the British writer FitzRoy Raglan and the Austrian psychoanalyst Otto Rank surveyed the legends of classical heroes and proposed prototypical heroic plots. Although their templates differed and they focused on different parts of heroes’ lives, they agreed that the story usually went like this: the male hero of myth is conceived miraculously, out-of-wedlock, or in other unusual circumstances. At least one of his parents is divine. After his birth, his caretakers abandon or hide him – sometimes in an attempt to kill him, sometimes for his protection. He is then adopted by animals or poor foster-parents. After an uneventful or arrogant childhood, he returns home. He kills an opponent, perhaps one or both of his parents, and maybe goes on to battle dragons, giants or other beasts. Eventually, he is crowned king.

It’s the sympathetic plot, pan-fried in Graeco-Roman gravitas.

In The Hero with a Thousand Faces (1949), the American mythologist Joseph Campbell outlines another spruced-up sympathetic plot. Reviewing fragments of myths and performances from around the world, Campbell argues that stories recount adventurous rites of passage. His template – the hero’s journey – is complicated, involving 17 stages organised into three broad phases, but he simplifies it as follows:

A hero ventures forth from the world of common day into a region of supernatural wonder: fabulous forces are there encountered and a decisive victory is won: the hero comes back from this mysterious adventure with the power to bestow boons on his fellow man.

A protagonist confronts obstacles, conquers them, and wins rewards: the sympathetic plot dressed in swashbuckling myth.

The sympathetic plot is ubiquitous. The late British journalist Christopher Booker reviewed 450 stories, spanning films, plays, novels, ancient epics and fairy tales. He organised them into his book The Seven Basic Plots (2004):

Overcoming the monster (Dracula; Theseus and the Minotaur) Rags to riches (Aladdin and the Enchanted Lamp; The Ugly Duckling) The quest (Aeneid; The Lord of the Rings) Voyage and return (Alice in Wonderland; Goldilocks and the Three Bears) Comedy/romance (Emma; Some Like It Hot) Tragedy (Faust; Romeo and Juliet) Rebirth (Beauty and the Beast; The Frog Prince)

With the exception of tragedies – we’ll get to them later – all are sympathetic plots. They tell of goal-directed protagonists who face obstacles, beat them, and revel in prizes. In fact, they differ almost solely on what the protagonist’s goal is, whether it be killing a monster, besting hardship, securing a priceless object, getting home, finding love, or escaping a dark spell.

What explains the appeal of the sympathetic plot? The answer, I believe, is entertainment. The sympathetic plot exhibits precisely those features required to suck us in and tickle our pleasure centres.

First, it intrigues us with obstacles. People feel suspense when they hear about someone who has trouble achieving a goal. Journalists exploit this psychological quirk with stories about toddlers stuck in wells and miners missing on mountains. The same mental widget is probably in play when we’re captivated by obstacle courses or competitive cooking shows. If you need someone to pay attention, tell them about a person who has difficulty getting what they want.

In 1989, the American psychologist Richard Gerrig used a simple experiment to show how obstacles create suspense. He reminded college undergraduates of familiar outcomes – such as that George Washington was the first president of the United States – then narrated the events leading up to the outcomes. However, for some participants, he emphasised the difficulties along the way, such as that the war left Washington frail and that people’s attention shifted to John Adams. Even though the students were reminded of the outcomes at the beginning of each story, they still felt suspense when they heard about the difficulties.

Why do obstacles intrigue? One possibility is that evolution has equipped us with psychological mechanisms for learning about people’s problems and how they solved them. If someone wants something and can’t get it, it’s useful to see which attempts worked and which didn’t. We profit from learning how people procured food, returned home, became prestigious or attracted a beloved’s affections. Armed with that information, we’re better prepared to face similar challenges.

The psychology of suspense explains why protagonists tackle problems. But what about rewards and ugly ogres? What about bullied kids and cheekbones of perfection? What about orphans? As with obstacles, mean opponents and impressive orphans keep us hooked. But crucially, they also interact to produce pleasure.

Sympathetic tales likely induce sympathetic joy, or the feeling of being happy for someone. Anyone who has cried happy tears during a movie knows how intense this pleasure can be. For the poor souls who don’t, here’s a letter that an Indian soap opera fan wrote to the show’s television network:

Congratulations on the wedding of Dr Ashwini and Badki. When the wedding was being telecast, my family and I could not control the tears of joy, and when the newlywed couple was blessed for the first time, our excitement knew no bounds.

We feel happy for those we want to help. People report greater sympathetic joy for their best friends, family members and fellow group members, compared with enemies, casual acquaintances and members of out-groups. Some scientists argue that these gratifying bursts urge us to assist others. Just as the pleasures of food and sex entice us, a beneficiary’s success feels good to motivate us to help them.

We’re more willing to help a friend who swerved into a deer than an anonymous drunk who crashed into a ditch

If stories induce sympathetic joy, then they should star the sort of characters whom we most want to help. But what should those characters look like?

According to many scientists, our psychology predisposes us to help family and people who will help us in return. Convincing someone that a character is a relation is tricky, especially when you’re telling the same story to several people. But convincing them that a protagonist is an appealing cooperative partner is much easier.

An appealing cooperator has two kinds of traits that offer unique benefits. On the one hand, cooperators might be speedy, gutsy or good with their hands. They might be a talented chef, an exceptional shot or full of useful information. Or they might have great cheekbones, in which case their friends get to hang with any hotties who come flocking. On the other hand, ideal cooperators are warm. They’re moral, generous and willing to stick around when things get hairy. A friendship is easier when the people involved want the same things; it becomes harder when their values diverge.

Aside from helping appealing partners, we also help people in need. Not arbitrarily, of course – our compassionate impulses are affected both by how close we feel to someone and whether we consider their suffering to have been controllable. We’re much more willing to help our best friend after she swerves into a deer than an anonymous drunk who crashes into a ditch.

In short, if you want to design a technology for stimulating sympathetic joy, present people with a character who is talented, honourable and pursues relatable goals, such as defeating an awful opponent. Characters should start out in need, but their suffering should be out of their control. Then have them succeed. In other words, present your audience with the sympathetic plot.

In his memoir On Writing (2000), Stephen King observed: ‘No one likes to root for a guy over the course of 300 pages only to discover that between chapters 16 and 17 the pig ate him.’ Yet Antigone, Romeo and Juliet, the good episodes of the TV series Black Mirror (2011-), and many other tales start out with the sympathetic plot only to end with the protagonist’s death.

An example of one such tragic tale is Kutri and Yai-mah, told by the Ainu. Kutri is not only handsome but also an inspired fisherman and an impeccable marksman. Meanwhile, Yai-mah – with her glossy hair and moon-round face – sings, plays the harp, carries heavy loads of wood, and prepares juicy fish-roe. When Kutri and Yai-mah first see each other, they are instantly enamoured. But both of them were betrothed as children to mean, ugly partners. They elope, but their exes find them. Kutri gets shot with a poison arrow and dies. Then Yai-mah kills herself.

Their story seems to violate everything I’ve just said. It introduces cartoonishly appealing characters who want to be together but can’t – and then kills them. Still, the tale ends with feelgood justice. After the lovers die, the ugly exes approach their hut. One of the exes kicks Yai-mah’s body, piercing himself with the poisoned arrow. The other ex burns down the hut, drawing the attention of the lovers’ dogs. They bite the arsonist and within days she becomes a rabid half-human, barking and scratching herself. Eventually, both exes roam jealous and mad through a demon-infested afterworld. Kutri and Yai-mah, meanwhile, live blissful afterlives in Sunday serenity.

Stories can do much more than tickle our pleasure centres: they can instruct, inspire, brainwash or bore

The point is that even violations of the sympathetic plot can offset the protagonist’s downfall with redemption. Romeo and Juliet doesn’t end with the star-crossed lovers’ deaths but with their families ending their feud and promising to honour the lovers with golden statues. After Antigone hangs herself, the son and wife of the king who ordered her death kill themselves too, leaving the king sorry and broken. In the film Titanic (1997), Rose’s lover Jack dies, but then the film reveals that Rose goes on to live a long, adventurous life, and is reunited with Jack in a wood-panelled afterworld.

Redemption softens the blow of sad endings. Yet it still doesn’t explain why plots take terrible, tragic turns.

One possible explanation is that stories can do much more than tickle our pleasure centres. They can instruct, inspire, brainwash or bore. They can reveal the strangeness of the mundane or the humanity of the other. They can critique fads by showing the awful outcomes they’re capable of spawning. Not only is a tragic ending sometimes necessary for these other-aims; it also reminds the audience that there’s more to a story than its gratifying arc.

For centuries, readers have debated the message of Romeo and Juliet. Some said it was about the destructiveness of hot passions, as if love were an enraged rhino set free in a ramshackle slum. Others read it as being about the powerlessness of love against destiny, as if destiny were the rhino and the lovers were the slum-dwellers trying in vain to fight it off. Regardless of who’s the rhino, the lovers’ deaths gesture at a larger meaning, inviting readers to decode it.

The sympathetic plot is universal, but that doesn’t mean it’s immutable. Like any other form of expression, the stories we tell reflect our values, memories and preoccupations. Take, for instance, the stories told in the West. As I write this, 16 of the 20 top-grossing films of 2019 are complete sympathetic tales. The four exceptions all resemble the sympathetic plot but stray in interesting ways. One tangles protagonist and antagonist (Us); another kills the protagonist for a greater good (Godzilla); and the last two end with twists, with one of them setting up for a sequel.

Yet people in the West today crave more than just the sympathetic plot. They want change, too.

Protagonists everywhere have external goals. They want to return home, conquer death, outsmart devils, and procure enchanted gizmos. Modern stories in the West retain the external goal, but they combine it with growth. Modern protagonists start out flawed. They’re stubborn, anxious, lazy or afraid. In chasing their external goals, they overcome their deficiency and become whole.

In The Wonderful Wizard of Oz (1900), Dorothy needs to get home. After fighting an evil witch and escapading through Oz, she comes to appreciate her loving family in Kansas. In the novel A Wrinkle in Time (1962), Meg travels through dimensions to save her father; in the process, she recognises her quirks not as faults but as handy assets. In the film The King’s Speech (2010), Bertie struggles to stop stuttering; he succeeds, but only after conquering his fear of failure. Without a protagonist who changes, stories seem amateurish, even broken.

Many writers consider internal change to be essential. In his story-treatise Into the Woods (2013), John Yorke writes: ‘Change is thus inextricably linked to dramatic desire: if a character wants something, they are going to have to change to get it.’ Lajos Egri is just as authoritative in The Art of Dramatic Writing (1946): ‘There is only one realm in which characters defy natural laws and remain the same – the realm of bad writing.’

It’s hard to identify when internal growth bubbled into popularity, but in The Origins of European Individualism (1995), the Russian historian Aron Gurevich points to the knightly romances of the 12th and 13th centuries. Whereas earlier tales emphasised monster-killing and wandering, knightly romances settled on a new narrative centre of gravity: self-discovery. The shift seems so prophetically modern that Gurevich’s descriptions of the stories sound like young adult novels for a Game of Thrones generation: ‘The heroes of such romances would be wandering knights, looking for adventures and exploits to perform, in which they might come into their own and find their true identities.’

Stories about internal growth emerged in the Middle Ages but, until the 19th century, only a few such tales trickled into the literary milieu. Then everything changed. The 19th century saw the rise of the Bildungsroman or coming-of-age novel. Change came gushing, and it flooded Western storytelling.

This fixation on personalities might explain why stories told in the West revolve around internal growth

The Bildungsroman is one of the pillars of European literature, if not its sturdiest and most ornate one. A protagonist sets out as an imperfect child and comes back a developed adult: that’s Candide (1759), Emma (1815), Little Women (1868-9), Adventures of Huckleberry Finn (1884), Native Son (1939), Invisible Man (1952), The Dharma Bums (1958), To Kill a Mockingbird (1960), Dune (1965), Song of Solomon (1977), Ender’s Game (1985), Norwegian Wood (1987), The Motorcycle Diaries (1995), the Harry Potter series (1997-2007), and so many more.

The Western focus on internal change makes anthropological sense: people in the West fixate on the internal. Psychologists have demonstrated this with a simple experiment. When researchers at Lewis and Clark College and the University of Nevada in Portland, Oregon, asked people to finish the sentence ‘I am ______’, people from the Western world responded with phrases such as ‘happy’, ‘a fun person’, or ‘a health-nut’. Meanwhile, rural Kenyans or Pacific Islanders said things such as ‘a mother’ or ‘a member of the Makea family’. Whereas most people define themselves by their relationships, people in the West emphasise their interests and personalities.

This fixation on personalities might explain why stories told in the West revolve around internal growth. What captivates people defined by their traits more than someone trying to conquer a flaw? And what can make them happier than to see that person succeed? The historical trajectory that set Westerners apart forged their distinctive stories.

Literary scholars today have forsaken massively comparative projects. Yes, they might contrast magical realism in Hebrew and Arabic literatures or look at avant-garde documentaries in Japan, France and the USSR. But the idea of scanning the world’s literature for echoes of a deep, thrumming human nature has become unpopular. Scholars probing for patterns are criticised for denying agency, ignoring history, and overlooking geopolitics. They’re condemned for diluting local flavours and projecting their assumptions about the familiar onto the foreign. As the American literary scholar Patrick Hogan observes in The Mind and its Stories (2003): ‘When universalism is mentioned in humanistic writing, it is most often denounced as a tool of oppression.’

This resistance is unfounded. Critics speak as if searching for patterns stands in Manichean opposition to appreciating particulars. It’s as if the two projects are children competing for a meal; by feeding one, we starve the other. But this is false. Rather, patterns and particulars are mutually informative. We appreciate the novelty of the Bildungsroman only when we compare it with the world’s literatures. Yet we can explain it only by peering into the cultural history of the people who told it.

The value of seeing particulars in a background of patterns comes across in our discussions of orphans. Audiences recognise that fictional orphans are common. And in explaining why, they often point to storytellers’ surroundings. People who notice orphans in Disney movies blame Walt Disney’s early years: soon after he and his brother gave their parents a house, a broken gas furnace killed their mother; the incident haunted Walt for the rest of his life. In The Maternal Voice in Victorian Fiction (1997), Barbara Thaden examined orphans in Victorian novels: because mothers had been reduced to such broken states, their child-protagonists could flourish only when freed from maternal destitution.

These explanations might be right (although I’m skeptical of both). However, each treats fictional orphans as parochial novelties. They fail to notice that parentless protagonists are everywhere – not only deeper in time than Disney or Victorian novels, but stretching from the Arctic to New Guinea, from Igboland in Nigeria to the Kachin Hills of Myanmar. Only by appreciating these threads can we finally recognise the social and psychological forces shaping the stories we tell.