“A drum! A drum! Macbeth doth come.” So say Shakespeare’s Weird Sisters in an uncanny prophesy of the Scottish play’s ubiquity this coming year. Rory Kinnear and Anne-Marie Duff star in Rufus Norris’s National Theatre production (previews from 26 February). Christopher Eccleston and Niamh Cusack are next in line in Polly Findlay’s RSC version (previews from 13 March). The Royal Opera, meanwhile, revives Verdi’s Macbeth (from 25 March) closely followed by Shostakovich’s satirical update, Lady Macbeth of Mtsensk (from 12 April). For good measure, Kit Monkman has made a fluidly poetic film version, opening in late March, to add to the growing catalogue of movie Macbeths.

I suspect there is an element of chance to this profusion of Macbeths. At the same time, there is clearly something about the play that speaks to us urgently today. It is fashionable to see King Lear, with its madness and nihilistic bleakness, as the most modern of Shakespeare’s tragedies. But you could make an equally good case for Macbeth.

It deals, after all, with the destructive nature of unchecked power-lust and political ambition: it is no accident that the play has spawned a host of modern variations, from a 1955 gangster movie, Joe Macbeth, to a 1967 parodic play, Macbird, in which Barbara Garson wickedly implicated Lyndon Johnson in President John F Kennedy’s assassination.

The play also provides one of the most extensive insights ever written into the mind and soul of a murderer. Malcolm refers to the Macbeths as “this dead butcher and his fiend-like queen,” but that’s not what we see on stage. Instead, Shakespeare’s play prefigures Dostoyevsky’s Crime and Punishment in giving us access to the mental agonies of a murderer who is simultaneously a free agent and a victim of fate.

Increasingly the play is also viewed as a study of inherited notions of manliness and of a marriage shadowed by the loss of a child: Justin Kurzel’s 2015 film, starring Michael Fassbender and Marion Cotillard, began with a funeral service for the Macbeths’ dead baby. It was palpably an event that shaped the subsequent action.

But I would argue there is one crowning reason, aside from its political relevance and domestic acuity, why Macbeth is always with us: it is quite simply the greatest theatrical poem ever written. The language, once heard, haunts the memory forever. Poets have also been quick to appreciate it. Peter Porter has said: “Nowhere is Shakespeare’s language more musical in the correct sense of the metaphor – not lyrical and euphuistic but continually thematic, with its own deep-set harmonies.” For a more detailed analysis, I would recommend the section in William Empson’s Seven Types of Ambiguity where he unpicks the lines “Light thickens / And the crow makes wing to the rooky wood.” Empson points to the suggestion of witches’ broth or curdling blood in “thickens”, to the cluck of the k-sounds implying an abrupt crackling of sticks and to the way the vowels at the end indicate increasing darkness as the crow takes flight.

Shakespeare may sometimes have been casual in his plotting but Macbeth proves that images and words were chosen with great deliberation. The idea of equivocation constantly recurs in a play in which “nothing is but what is not”. Frank Kermode says: “It is surely impossible to deny that certain words – time, man, done – and certain themes – blood, darkness – are the matrices of the language of Macbeth. One might guess that they took possession of Shakespeare as he did his preparatory reading. That they are thereafter used with conscious intention and skill seems equally certain.”

Precisely because it is an intricate dramatic poem, the play has had a somewhat chequered stage history. In the Victorian theatre, with its love of vast sets and hordes of extras, spectacle swamped action. As late as 1930, James Agate, writing of Gielgud’s Macbeth, complained of the look of the thing: “The Macbeths and the Macduffs, the Banquos and the Malcolms are – and must always be – a forbidding lot with their tartan manners, berserk headgear and uncouth whiskerage.”

Even the first production I saw, at Stratford in 1955, was a broken-backed, two-interval affair. What it did contain, aside from Vivien Leigh as a wonderfully viperish Lady M, was one of Laurence Olivier’s greatest ever performances as Macbeth: I still recall his guilt-haunted look in the early scenes, the comic irony with which he manipulated Banquo’s murderers and the spiritual desolation of the climax in which he lamented the loss of “honour, love, obedience, troops of friends” (his voice actually soared on “troops”) among the many things which should accompany old age.

Since then I’ve seen countless productions that have proved one thing: that Macbeth gains enormously from intimacy and speed. The breakthrough came with Trevor Nunn’s version at Stratford in 1976 staged in a derisory tin hut called The Other Place. The action, played without interval, took place within a white circle that became an occult cipher and a space enclosing inner terror. We ourselves became complicit in Judi Dench’s invocation of demonic spirits as Lady Macbeth; in the jaw-slackening disintegration of Ian McKellen’s Macbeth as he confronted Banquo’s ghost; in the poleaxing grief of Bob Peck’s Macduff as he learned of his family’s slaughter.

It was a production that left its mark on many of its successors, including Greg Doran’s in 1999 with Antony Sher and Harriet Walter; Rupert Goold’s in 2007 with Patrick Stewart and Kate Fleetwood; and that of Kenneth Branagh and Rob Ashford staged in a deconsecrated Manchester church in 2013.

But, while small spaces help release the play’s poetic energy, there is room for alternative versions. In particular, foreign-language Shakespeare, which is not bound by the original text, sometimes offers valuable insights. No one who saw it will ever forget Yukio Ninagawa’s 1985 Japanese production, which returned to the Barbican this year. Setting the play in the world of 16th-century samurai warriors turned it into a poetic meditation on human transience: two withered crones parted wooden screens to reveal images of extraordinary beauty such as the cherry blossom that fell in abundant profusion during the stylised battle scenes. A play that is traditionally shrouded in darkness and gloom suddenly became a miracle of colour and light.

More often Macbeth has a nightmarish ambience, which is one reason it has proved irresistible to movie-makers. Top of my list would be Orson Welles’s low-budget 1952 film in which, as David Thomson pointed out, “Welles makes a noble man who has sold his soul” – a reminder there should always be something of the fallen Lucifer about Macbeth. Akira Kurosawa’s 1957 Japanese version, Throne Of Blood, is famous for its samurai vigour and misty castles, but he translated the action to a period when regicide was less a transgressive act than a routine occurrence. Roman Polanski’s 1972 film contained few memorable performances but was full of intelligent touches such as the final suggestion that Malcolm’s reign would be as much shadowed by the supernatural as Macbeth’s. And the recent Justin Kurzel version, with its fierce armies clashing on a darkling plain, returned the play, in spite of an international cast, to its historic Scottish roots.

If Macbeth continues to obsess us, it is because it seems both modern and timeless. We see all around us today the corrosive folly of overweening ambition and the insecurity that breeds tyranny. But Macbeth can never be reduced to a set of moralistic, crime-does-not-pay platitudes. It is, beyond all else, a supreme theatrical poem that has a language that eats into the soul. It also fulfils the definition of a dramatic masterpiece in that, as we hope to discover in early 2018, it is capable of infinite renewal.

‘Avaunt and quit my sight! Let the earth hide thee!’ Laurence Olivier as Macbeth at the Old Vic, 1937 or 1938. Photograph: Culture Club/Getty Images

Potted profile

Date written: Assumed to be 1606, given a cryptic reference to a Jesuit priest, Father Henry Garnet, arrested for his part in the gunpowder plot of November 1605 and author of A Treatise of Equivocation.

Plot: Spurred on by the Witches’ prophesies and the promptings of his wife, Macbeth murders the Scottish king, Duncan. To secure his throne, Macbeth launches a reign of brutal tyranny before being killed in his castle at Dunsinane.

Sources: Holinshed’s Chronicles gave Shakespeare the main story of Macbeth and Duncan. The play may also have been inspired by the fact that James VI of Scotland, who took over the English throne in 1603, claimed one of Macbeth’s victims, Banquo, as his ancestor.

Superstitions: Many actors, believing the play to be cursed, refuse to name it (always referring to it as “the Scottish play”) or to quote from it outside the rehearsal rooms. Countless injuries sustained in performance, affecting Laurence Olivier, Diana Wynyard and Charlton Heston among many others, may be more due to the fact that the play is often performed in semi-darkness rather than to any inbuilt curse.

Best production: Trevor Nunn’s, starring Ian McKellen and Judi Dench, at Stratford’s The Other Place in 1976.

Worst performance: Peter O’Toole’s at the Old Vic in 1980, which was savaged by the critics, who dubbed it “Macdeath” and “Macflop”, before it went on to attract big audiences.