PUTTING on a musical is a high-stakes gamble. Big hits run for decades and can make billions, but flops fail fast. Only one in ten musicals will turn a profit; two will lose all their money. Investments aren’t small, either: West End musicals cost an average of £6m-7m to stage. A new production that fails to start strong must turn things around quickly. In mere months—sometimes less—many cannot keep up with running costs and must be scrapped. The formula for success is so elusive that a great show is often described as “magic”. But behind the illusion, there are tricks of the trade that help a West End run run. When looking for the secret to success, the 1980s is the place to start. Perhaps there was something in the hairspray, as the West End’s five longest-running productions premiered between 1981 and 1986. The highest charting, “Les Misérables” and “The Phantom of the Opera”, are still running now. These were the first mega-musicals; special-effect-studded extravaganzas integrating new technology, lighting and sound systems. The chandelier in “The Phantom of the Opera” shocked and amazed as it swung through the audience and smashed onto the stage. Hydraulic ramps in “Starlight Express” sent the technicolour roller-skating locomotives to even dizzier heights. This era forever changed audience expectations, and drove up the cost and risk for future productions. Special-effect spectaculars can also fail—the “Lord of the Rings” musical cost a staggering £25m to produce (the West End’s most expensive production) but only managed to scrape a year on the precious stage before being sent back to Middle-Earth.

Some names are synonymous with musical-theatre success, particularly Cameron Mackintosh and Andrew Lloyd Webber. Mr Mackintosh has produced the three longest-running musicals, and Andrew Lloyd Webber composed one-quarter of the top 20 longest-running musicals (plus some that didn’t make the list including “Joseph and the Amazing Technicolour Dreamcoat”). Yet even these two legends are far from infallible. Mr Mackintosh’s bizarre and bawdy take on “Moby Dick” closed after four months in 1992. And before the launch of “Stephen Ward” (2014), based on a British establishment sex-scandal, Mr Lloyd Webber said, “I haven't had a hit in 20 years. I've written six musicals in that time. I'm resigned now to the fact that anything I do probably nobody is going to like.” The remarks were prescient; “Stephen Ward” lasted four months before an early close.

Musicals with some familiar element hedge the risks. “Mamma Mia”, a hit of the late 1990s based on the songs of ABBA, was the earthquake before the tsunami of “juke-box musicals”. “Thriller Live” (Michael Jackson), “Sunny Afternoon” (The Kinks), “We Will Rock You” (Queen) are a handful significant of an enduring trend. Wikipedia lists 58 juke-box musicals released since 2000: roughly 18 of them have made it to the West End.

The juke-box musical seemed like a bulletproof approach: the big hits of a global band can be appreciated regardless of a viewer’s native language, and one-third of the West End's ticket-buyers come from abroad. Yet there have been some big disappointments. “Viva Forever”, based on the music of the beloved Spice Girls, lost £5m and closed within a year. In 2007 “Desperately Seeking Susan”—combining Blondie’s music with a hit 1985 film—couldn’t be saved. One critic wrote that “Susan has me desperately seeking the exit.”

Plenty of movie and book spin-offs have had better luck. “The Lion King”—based on the adored Disney classic—sits at number seven on the top 20 longest-running list. “Matilda” won awards with a story by Roald Dahl, and this year’s Olivier winner for Best New Musical was a real-life story turned feel-good film turned musical, “Kinky Boots”. Others have been more poorly judged; “Carrie”, based on Steven King’s horror novel about a teenage girl with anger issues and telekinetic powers, was a legendary bloody disaster.

Musical creators seek to minimise the gamble of producing a show, so it is no surprise that the audience, too, tries to minimise the gamble when picking one. A West End night out is a rare occasion for most, and seeing an adaptation of a well-known film or a musical based around familiar music feels safer when forking out for tickets (the best seats can fetch over £100). This risk-aversion also explains the enduring success of the 1980s big hitters. They’re tried and tested.

But the industry and the audience can still surprise. Take “The Book of Mormon”, a comedy about missionaries, written by the makers of “South Park”. It parodied the musical genre—attracting a different crowd to the seats. It also created the hype of the “hot ticket” through social media; the promise of a good night out no longer comes only from critics or billboards. Big budgets, industry experts, famous soundtracks and familiar characters are always going to help a musical’s chances, but there is no flawless formula. You still need some magic.