With Hollywood spending millions on new versions of age-old characters, from Frozen’s Snow Queen to Cinderella, fairytales are more popular than ever. But they’ve had to adapt, with lots of dark twists and no more sweet, biddable girls

“Forget villain,” the producers of Frozen declared when translating Hans Andersen’s The Snow Queen to the screen. The icy enchantress, who enthralled me to the core of my being when I was young, has been utterly transfigured – softened, redeemed – in Jennifer Lee’s screenplay; she becomes Elsa, an intense young woman with paranormal gifts of “cryokinesis”, which she cannot control. No blame, no villainess. The snow queen, who in Andersen’s extraordinary story steals away Kai’s heart after a shard from the trolls’ accursed mirror enters his eye, has metamorphosed into a vulnerable girl, saved by the love of her sister, Anna.

Frozen has become the most successful animated film of all time: Anna and Elsa dolls have overtaken Barbie in sales for the second

Christmas running, a jolly seasonal short has been made called “Frozen Fever”, and the studio is assumed to be planning a sequel.

It isn’t, of course, the first time a classic fairy story has been seized, disarticulated and recast, its apparently defining elements radically altered. The sequence of twists and variations it has undergone reveals how fairytales respond to social values and needs over time. For instance, female screenwriters and directors, whether working in Hollywood or arthouse cinemas (and taking their cue from women artists and writers), are making a difference. Since the 1960s, criticism has pointed to the lies peddled by stories such as “Beauty and the Beast” – especially the overvaluing of wealth and power invested in the male – and as a result, the generations brought up by grandmothers and mothers who know their Betty Friedan, Angela Carter and Naomi Wolf have brought a new consciousness to the way classic stories are reshaped for the screen.

A screenwriter such as Linda Woolverton (whose credits include Beauty and the Beast as well as many other hits, not least Maleficent, this year’s updating of Sleeping Beauty), is a self-declared second-wave feminist in the American mould. Meanwhile, beautiful and brilliant divas, now entering their middle years, are relishing the chance to play the towering villains of fairytale: Helena Bonham Carter threw herself into the screeching termagancy of the Red Queen in Alice in Wonderland (again, a huge box office success) and she’ll be appearing next year in Disney’s new Cinderella, this time in the benign role of the fairy godmother. Will clever Cate Blanchett, in the part of the wicked stepmother, find unexpected reserves of humanity in this famous villain? Kenneth Branagh is directing, but industry gossip reports many comings and goings before he was in place. As in the fable of the robbers who fall to murdering each other for the loot, the extraordinary riches to be made from these films leads to prolonged quarrelling, and sometimes – as in 2012’s Snow White and the Huntsman – the storyline limps along across terrain strewn with crowd-pleasing pieties.

Fairytales are enjoying a huge rise in popularity and influence in symbiosis with the internet. The traditional functions of the bard and the griot in predominantly oral cultures included “keeping the memory of the tribe”, as Derek Walcott remarked in his Nobel speech, and the enchantments of technology have placed the power to do this in our hands. The entertainment industry increasingly harvests the common store of fairytale to develop one vastly expensive vehicle after another to reach the global market. Some are disastrous (Hansel & Gretel: Witch Hunters), but at the same time, this new ocean of story provides film-makers and website builders of slender means with magnificent new opportunities.

Like a mother tongue, the stories are acquired, early, to become part of our mental furniture (think of the first books you absorbed as a child). The shared language is pictorial as well as verbal, and international, too. Such language – Jung called it archetypal – has been growing into a common vernacular since the romances of classical antiquity and the middle ages – Circe from the Odyssey and Vivienne from Morte d’Arthur are recognisable forerunners of fairy queens and witches, and the sleeping beauty herself first appears in a long medieval chivalric tale, Perceforest. A fairytale doesn’t exist in a fixed form; it’s something like a tune that can migrate from a symphony to a penny whistle.

Facebook Twitter Pinterest Helena Bonham Carter as the Red Queen in Tim Burton’s Alice in Wonderland. Photograph: Alamy

Or you can compare it to a plant genus, to roses or fungi or grasses, that can seed and root and flower here and there, changing species and colour and size and shape where they spring. But if the prevailing idea of an archetype gives too strong an impression of fixity, the picture-language of fairytale is fluid and shapeshifting: a rose is not a rose, an apple not an apple; a princess or a villain signify far more than what they seem. A dictionary of fairytale would look more like a rebus made up of icons: snow, crystal, apples, dark forests, pinnacled castles, mermaids, toads, giants, dragons, sprites, fair princesses, likely lads and crones.

The symbolism comes alive through strong contrasts and sensations, evoking simple, sensuous phenomena that glint and sparkle, pierce and flow, by these means striking recognition in the reader or listener’s body at a visceral depth (gold and silver; diamonds and rubies, thorns and knives; wells and tunnels). It’s an Esperanto of the imagination, and it’s available for any of us to use – in almost any medium. Whether in Hong Kong or Tasmania, the comic genie of the lamp, irrepressibly voiced by Robin Williams in the film of Aladdin, can speak to anyone.

When I allude to “The Little Mermaid” or “Bluebeard” or “Cinderella”, we know where we are. This knowledge excites a desire to know more and know it differently, to detect the deeper meaning behind the tale, to see, as the American writer Paula Fox says, that “the meaning of one thing could also be the meaning of a greater thing”. A consequence of this has been a spate of anti-tales and countertales, inversions and twists, in which writers or film-makers enter the well-known story from a new angle, in order to refresh it.

When writing a new version of “Hansel and Gretel”, Michael Morpurgo recalled his early experience of the tale: “I was about six years old. I knew the story already. I think my mother must have read us the Brothers Grimm version … I remember that I identified particularly strongly with Hansel. At the time I was just beginning to comprehend what it meant to be a stepchild. The truth is that I am still trying to come to terms with it … It’s been a troublesome old gremlin.” Morpurgo’s mother had fallen in love with another man while his father was away during the war: “Let’s just call it collateral damage of the domestic kind,” he says. Fairytales are expert in that damage, ordinary human suffering, and they offer many and varied ways of resistance and hopes of survival: revenge fantasies (killing the witch), dreams money can buy (rags to riches, as in “Cinderella” and “Aladdin”), mockery and jokes and satire and exaggeration (Giambattista Basile’s The Cinderella Cat, or Madame d’Aulnoy’s The Ram).

Walter Benjamin singled out fairytales for their “cunning and high spirits” with which the oppressed countered oppression. Fairytales imagine possibilities of transformation: the beast loses his monstrous shape and with it his monstrous nature, or, as in Angela Carter’s scintillating piece of erotica, “The Tiger’s Bride”, beauty becomes a beast herself and luxuriates in it. Above all, “the Fairy way of writing”, as Dryden described it, offers a way of transforming tradition. Working with a plot, a character, images and motifs already familiar to the intended reader or audience gives freedom to retaliate, protest and reinvent. Often the possibility involves a novel metamorphosis taking place inside the well-known story: the American artist Kiki Smith made a lifesize sculpture, called Daughter, of a furry-faced child in a cloak – the offspring of Red Riding Hood and the wolf. The poet Robin Robertson, in an extraordinary reimagining of selkie lore, “At Roane Head”, enters the figure of the bridegroom who puts aside his sealskin to live and have children with a human. His own imagination is freighted with the fairytales of Scotland as well as classical myths, and he sees the tales as dream work. Dreams of such beings from other “secondary worlds”, beneath the sea or from the sky, fill the stories of the Arabian Nights as well as Dvořák’s opera Rusalka and “The Little Mermaid”. Robertson says that for him, “true poetry moves easily between the two – dream and myth – feeding and pollinating”.

An alternative term for “fairytale” is “wonder tale”, from the German wundermärchen, which catches a quality of the genre more eloquently than “fairytale” or “folk tale” because it acknowledges the defining activity of magic in the stories. The suspension of natural physical laws produces a heightened and impossible state of reality, which leads to wonder, astonishment, the ’ajaib (astonishing things) sought in Arabic literary ideas of fairytale. The language of fairytale doesn’t belong to literature alone: it is a lingua franca for expression in art and music, theatre and opera, fashion and even architecture (Dolce & Gabbana’s recent collection, “Enchanted Sicily”, teemed with elves and pixies, friendly forest creatures and allusions to Tchaikovsky; Grayson Perry’s folly, newly built in Essex for Alain de Botton’s happiness project, is a kind of jewel casket-cum-forest hermitage cum-Baba Yaga cottage). CS Lewis once remarked that a fairytale did not have to be especially well written to be memorable, and he’s right: there are oddly few literary masterpieces among fairytales. Yet conversely, there are very few great writers or even composers who don’t draw on fairytale, from Shakespeare to Dvořák, Hans Werner Henze and Judith Weir.

The social context keeps making a difference, though, and the stories have to be updated accordingly. Sweetening and lifting the material were the preferred modes of the past, as in George MacDonald’s The Light Princess or Jacques Demy’s 1970 film Donkey Skin, which treats father-daughter incest with ironic lightness. But this approach has given ground. No more sweet, biddable girls: in family entertainment, heroines have become fast-talking, athletic and indomitable; they take on all-comers, especially would-be lovers, and they show no sign of falling in love, let alone into a tender reverie as in the bad old days of Disney’s Cinders.

Facebook Twitter Pinterest Angelina Jolie in Maleficent. Photograph: c.W.Disney/Everett/Rex

“I know, she’s going to wake up now!” said the little boy in the front row at a screening early this year of Maleficent – the young prince had begun tremulously approaching the lovely young girl laid out on her bed as if on a bier. Like the boy, the audience knows the story: the heroine will be woken from 100 years of charmed sleep by a kiss and will marry her prince and live happily ever after. But the Disney spectacular springs a surprise: the kiss fails.

The climactic scene of the “Sleeping Beauty” story sounds like date rape today, so Maleficent had to change the way it’s played. Again a supposed intrinsic, archetypal element of a familiar fairy story has been radically altered. In the tale’s 17th-century Neapolitan predecessor, the sleeping beauty gives birth to twins after the (married) king comes upon her and “plucks the fruits of love”. The famous Perrault version of 1697 dropped a blackout blind over this. By contrast, when Burne-Jones painted his voluptuous cycle Briar Rose he created a vivid and perverse tableau of late Victorian sexual anxiety, depicting many suitors who hang bleeding on the roses in the hedge of thorns after they have attempted to reach the sleeping princess. Woolverton’s screenplay works hard to transform the archetypal wicked older mother figure. Angelina Jolie is the film’s producer, and in real life, as we know, she has herself adopted children as well as had some of her own, and consequently has a personal stake in revising the figure of the bad stepmother in fairytale. The happy ending of Maleficent took me by surprise and runs against the current trend towards fairytale darkness, shining a beam of security and love on the increasingly common – and fraught – relations of step-families today.

A less market-savvy approach, more arthouse than Pixar, opens up a story from another angle, often inverting villains and heroes: the novelist Dubravka Ugrešić in her novel Baba Yaga Laid an Egg identifies her alter egos in the story with the three witches as well as the terrifying old hag of the Russian fairytale forest; Neil Gaiman, in the marvellously shivery story “Snow, Glass, Apples”, sees little Snow White through the eyes of the wicked stepmother, and she’s a bloodsucking fiend: “I do not know what manner of thing she is. None of us do. She killed her mother in the birthing, but that’s never enough to account for it … If I were wise I would have killed myself before ever I encountered her.”

The Grimms’ plots as well as their stories’ stark mood have come to define the fairytale itself as a genre. Yet, in the 19th century, when Joseph Jacobs collected “English” folk tales and fairy stories from all over the archipelago, he was committed to local distinctiveness, as was Giuseppe Pitrè in Sicily or Alexander Afanase’ev in Russia, and their collections include a lot of high spirits, elaborate flights of fancy and colourful language. Every culture was soon busily following suit, recording and setting down the folklore of their ethnic and sometimes linguistic heritage – in ballads, poems, songs, stories. By the beginning of the 20th century, the trend against national particularity and towards universal homogeneity had set in: the Scotsman Andrew Lang with his coloured Fairy Books (Green, Rose, Yellow, etc) collected tales from all over the world, ran them through an editing process that made them sound very alike, and in ways that emphasised the common heritage of humanity.

With the coming of huge mass markets, the problem of fairytales’ collusion with power grew deeper. In the late 1970s, they began to grow up and address adults again, with Anne Sexton’s mordant elegies, Transformations, and Carter’s The Bloody Chamber and The Sadeian Woman. No longer children’s fare, the tales were opened up to reveal their harsh, unflinchingly realist inner core – in, for example, the renderings of a fellowship of writers centred on Emma Tennant’s literary magazine Bananas (including Michèle Roberts and Sara Maitland, as well as Carter, Terri Windling and Jane Yolen). The writers – chiefly but not only female – were demanding that women have a voice in the stories’ disclosures, and they are still putting new wine in old bottles, in Carter’s phrase, to watch them explode. The activity is growing, for example in the fiction of John Burnside, Ali Smith and Helen Oyeyemi, and in the productions of Marianne Elliott and Melly Still for the stage.

Facebook Twitter Pinterest “The Sleeping Beauty”, 1873-94 by Edward Burne-Jones. Photograph: AKG-images

Fairytales have lost the innocence that Dickens praised in them so trustingly. Hollywood may want to forget villainy but elsewhere, the dreams are no longer rainbow-coloured, the wishful thinking no longer starry-eyed. “The reign of fairytales ... could be ecstasy,” the Italian writer Cristina Campo has noted, “but it is above all a land of pathos, of symbols of pain”. Pornography, once rather closely sequestered in a niche for the avant garde and/or the dirty mac brigade, now colours the enchanted castles and mysterious forests: in the film Sleeping Beauty, directed by the writer Julia Leigh and produced by Jane Campion, the heroine volunteers as a sex worker in a specialist brothel for high-class customers. Gothic, once an independent and exclusive genre, populated by characters of its own (vampires, zombies), has spread its barbed and shadowy wings over fairytale. Fifty Shades of Grey patently revisits “Bluebeard”, and although it is extremely hard to see what the reasons are for the book’s extraordinary success, it certainly provides evidence of a widespread naturalisation – normalisation – of sadomasochistic fantasy. But even in the family-friendly (“mild violence”) ambience of Maleficent, S&M tinges the fangs and python-horned turban of the bad fairy. Some of the writers and artists who are wrapping fairytales in darkness are keeping to the spirit of the form as a truth-telling conduit, passing on the hard facts of life in deceptively thrilling and enchanted disguise. One of the best films of recent times, Biancanieves, directed by Pablo Berger, revisits Disney’s Snow White and transforms it with visionary energy. But the ending is not happy; that would be a step too far. Fairytales are drawing closer to myths, where dreadful fate is seldom averted, and the lessons the hero or heroine offer give little consolation.

Richard Dawkins was hounded by the press this summer when he supposedly attacked fairytales on the grounds that they teach children to believe in magic and the supernatural. Paradoxically, magic has become so vigorous in contemporary fantasy because science has made it possible to realise many of its inventions on screen: long before CGI brought into being fantastic monsters and pixies, or made vanishing and flying almost banal, the imagination of storytellers had conjured them into existence. Technology follows fantasy, not the other way round.

Impossible – absurd – enchantments define fairytale as a form of storytelling, but the magic also gives expression to thought-experiments: the wicked fairy turning out to be capable of love, the Frozen princess thawed into humanity by her heroic sister’s staunchness and love. Fairytale is a country of the mind made by imagery, by riddles and charms, spells and nonsense; it uses language to create imaginary structures in which language itself is supremely powerful: Rumpelstiltskin is undone when the heroine discovers his name.

Disquiet about fairytales has become rather more widespread, for reasons distinct from Dawkins’s hesitations: it arises from the recognition of fairytale territory as that land of pain that Campo evokes. Dislike of shallow promises and easy solutions in times of war, eco-disaster and other horrors have grounded fairytales; the escapist stories have become lenses through which difficult truths are inspected. Children around the world continue to grow up with the magic of fairytales in books, and to relish the multiple ways they are adapted, updated and put on to stage and screen. But the “realisation of imagined wonder”, which JRR Tolkien saw as the aim of the genre, isn’t always bright and shiny any more; its skies have clouded over.

• Marina Warner’s Once Upon a Time: A Short History of Fairy Tale is published by OUP.