A creepy clown has just chomped a monstrous chunk out of the box office – and It is not alone. This year, horrors Annabelle: Creation and the fantastic Get Out have thrived while other genres struggled to draw in the crowds. The Stephen King adaptation about the evil Pennywise has made nearly $180m (£134m) worldwide in its opening weekend, the biggest-ever opening for a horror. It’s official: horror films are firmly part of the mainstream – for good or bad.

The Conjuring “universe” kicked off in 2013, buoyed by the success of the Insidious series. The supernatural sequels and spin-offs that followed its success have since magicked themselves into the worldwide top 30, a list that’s more typically dominated by action, sci-fi and family films. Annabelle and Annabelle: Creation were both part of this phenomenon – the latter made it to number one at the US box office – and there’s more to come: there are spin-offs called The Nun (scarier than it sounds) and The Crooked Man in the offing, alongside The Conjuring 3. It seems audiences cannot get enough.

Facebook Twitter Pinterest Lili Taylor in The Conjuring. Photograph: New Line Cinema/Sportsphoto Ltd./Allstar

So: why the stampede? Over to Ian Sandwell, box-office reporter at industry mag Screen International. “There’s always a core audience willing to turn out for a horror, and if it gets good reviews, it can cross over into the more general Friday-night crowd. Both Annabelle: Creation and It had built-in audience recognition, so that doesn’t hurt. It is a genuinely good movie that’s received some of the best reviews for a mainstream horror in some time. It’s led to It being a film that can appeal across different audiences, from the generation terrified by the 1990 adaptation to teenagers, for whom It might be their first-ever horror.”

It review – a feast of scary Stephen King, plus the haunted kitchen sink Read more

Cinema staff have noted the variety of punters pouring in to see It. Picturehouse’s Clare Binns points to the nostalgic influences in the 80s-set film. “It’s really a Spielberg/Lucas film for those that were too young for when those directors were cranking out their most iconic films. Stranger Things led the way.”

Most contemporary horrors owe a debt to more sinister classics. Period horrors such as Annabelle have taken visual and thematic cues from supernatural 60s and 70s horror, while Get Out put a modern, racial spin on the genre with a Stepford Wives vibe, also owing a debt to Rosemary’s Baby. The latter is echoed in Darren Aronofsky’s new film Mother!, out this weekend.

According to Alan Jones, director of London festival FrightFest: “Horror is a cyclical genre. There’s always one movie that kicks off another golden era. First it was Hammer, then Psycho, then Rosemary’s Baby, then The Exorcist, The Texas Chainsaw Massacre, Halloween, A Nightmare on Elm Street, Scream, Blair Witch, Hostel, Paranormal Activity, Saw, and now it’s The Conjuring universe and It.”

In terms of box-office success, the movie world is see-sawing between sub-genres such as slasher, psychological, paranormal, teen, comedy-horror, found-footage and “torture porn”. Remakes of Asian films have also made a significant dent in the box office: The Ring and The Grudge both place in the top 30 horrors of all time. And just as Last House on the Left and Night of the Living Dead helped popularise the genre in the 60s and 70s, indie hits still give studios fresh encouragement to produce mainstream horror.

Facebook Twitter Pinterest Mia Farrow in Rosemary’s Baby. Photograph: Allstar/Paramount

The rule can also work in reverse. “There’s a reason studios churn out horrors that are basically just loud noises without any thought to atmosphere,” says Sandwell. “It’s because they generally pull in an audience on the opening weekend at least. But maybe It, which balances jump scares with likable, well-rounded characters and a genuinely creepy atmosphere, could lead to studios taking more of a risk when it comes to what constitutes ‘mainstream’ horror.”

While a film based on jump scares and young kids bonding seems relatively benign, the rise of so-called torture porn raises more troubling questions. I’m a big fan of paranormal horror but found the extremes of Hostel and Saw, which contain extended torture scenes, the most challenging part of my otherwise rewarding job as a film critic. A teenage horror fan once told me: “It’s fine if you don’t imagine it is happening to you.” Aren’t we supposed to be identifying with the characters we see on screen? And if we’re not, are we becoming dangerously desensitised?

Alice Haylett Bryan, a lecturer in film studies at King’s College, London, says: “Certain subgenres of horror are undoubtedly getting more extreme, but this is the case across culture as a whole, with computer games and television programmes such as The Walking Dead. We are now living in an age where real acts of violence, and indeed death, have been screened on Facebook and YouTube. Could it not be argued that this desensitises viewers on a more fundamental and concerning level?”

Facebook Twitter Pinterest Millie Bobby Brown as Eleven in Stranger Things. Photograph: Wenn

If she’s right, then perhaps this overall desensitisation is making younger people bolder in their movie choices. “I see horror constantly regenerating and attracting a much younger audience who can now access content in the way we never could, and are getting intrigued by what the genre possibilities are,” says Alan Jones. The good news for cinemas is that today’s teens and twentysomethings watch horror on the big screen as well as on streams or downloads. Student Andrew Williams, 19, recently went to see It with his friends. He says: “The best horror movies need to be watched in total darkness with surround sound and silence, but with other people around you so that there is an atmosphere of fear in the room.” As for choosing what to watch: “We decide as a group. Many of my friends don’t love horror, so it has to be enticing.”

With easy access to trailers and online reviews, it’s easy to see how recent generations may be persuaded to shell out for cinema tickets when their parents and grandparents would have opted for DVD or video. Williams’s experience also highlights the influence of peer pressure: young people have always been more inclined to use cinema outings as social experiences and may be led by dominant members of their social group. Mike Berry – a clinical forensic psychologist at the Royal College of Surgeons in Ireland, says: “Most people go in small groups to the cinema. Individuals apparently enjoy horror films more in the presence of a stronger character, and less when with someone else who is weak and frightened.”

It’s not just age that’s altering: Jones says the gender gap is narrowing. “The female demo at FrightFest has grown consistently over the years: now I’d say it was roughly 60% male, 40% female.” Picturehouse’s Binns,says: “Women are now just as keen to see and openly enjoy what was seen in the past as a ‘boys’ subject matter’.” Steve Oram, writer and star of British horror comedy Sightseers, suggests: “Horror has a much wider gender appeal these days. It’s less for sweaty teenaged boys in shorts. And that’s a good thing!”

All in all, horror looks on the way to becoming cinema’s most reliable profit-generator. But surely not everyone likes scary movies? Perhaps the typical traits of a horror fan are more common than we think these days. Berry says: “There is a significant correlation between people who accept rule-breaking behaviour and have an interest in horror movies. People who watch horror films want to be frightened, excited and scared, yet psychologically distance themselves from the reality.”

Facebook Twitter Pinterest Petr Janis and Jay Hernandez in Hostel. Photograph: Allstar/Lionsgate

That’s a change from the moral panics of the 80s and 90s. “Twenty years ago,” continues Berry, “it was suggested people who loved the blood and gore displayed low empathy, with males often strongly identifying with the villain while ‘thrill watchers’ displayed empathy with the victims, and liked the suspense of the film.”

Berry also cites Carl Jung on the genre’s more general appeal. “Apparently, he believed horror films ‘tapped into primordial archetypes buried deep in our collective subconscious – images like shadow and mother play an important role in the horror genre’. Others have argued that people watch violent horror films to relieve themselves of aggressive feelings.”

Haylett Bryan points to the wider meaning of horror booms. “The political and cultural climate undoubtedly influences both the production and reception of horror cinema. The success of Get Out and its exploration of the fetishisation and exploitation of the black body is testament to that. How horror cinema responds to the Trump presidency through the types of film produced, for example, will be very interesting. But across international horror as a whole, over the last 10 years or so, the rise in far-right political support has been reflected in narratives that allegorically or directly explore race, immigration and nationhood.”

Facebook Twitter Pinterest Lulu Wilson in Annabelle: Creation. Photograph: Justin Lubin/AP

Gareth Tunley, who directed the British psychological horror The Ghoul, says: “Horror writers have to tap into the subconscious in order to access anything remotely scary, so exploring the things the rest of society would prefer swept under the carpet or locked up in the woodshed goes with the territory.”

So is there something about our current society that is driving people into the cinemas, seeking scares? Stephen Prince is professor of cinema at Virginia Tech. “Whether or not the world is a more dangerous place today than in earlier eras is debatable. How many of us, for example, would choose to live in a time before antibiotics? But people today may feel they are under greater threats than ever before. Part of this is due to the interconnected world - the internet magnifies a sense of peril. More people in US die from car accidents than from terrorist attacks, but terrorism seems much scarier than cars because it gets plenty of news coverage. Horror films address questions that continue to haunt us, and these have to do with the frailty of the human body, the fragility of our hold on life, and the ambiguities about what a human being is and why people can’t be trusted to remain reliably human.”

As for the opening weekend of It, Margee Kerr, a sociologist and the author of Scream: Chilling Adventures in the Science of Fear, takes a topical approach. “After weathering, literally and figuratively, two hurricanes and a political climate to match, we’re all ready to escape reality for a while and fight a monster we know we can beat. There is a real sense of satisfaction when our fears are made real, at least real on the big screen, and we can celebrate their defeat.”

“Maybe the worse things get in the real world, the more we need the emotional release of being scared half to death,” says Tunley. “When you’re living in a world which may end in nuclear fire at any point, a bit of casual onscreen dismemberment seems an appealing bit of light relief.”

Send in the clowns.