He has written perhaps the greatest canon of pop music, collaborated with everyone from Elvis Costello to Rihanna, and even soundtracked a video game, but at 77, Paul McCartney is ticking off another career first: writing a musical.

His first stage musical is to be an adaptation of Frank Capra’s 1946 film It’s a Wonderful Life, and will open late in 2020. McCartney will write music and lyrics for the project. Lee Hall, author of Billy Elliot and screenwriter for the recent Elton John biopic Rocketman, is also writing lyrics, and the show’s book.

The musical was initiated by Bill Kenwright, the British theatre producer known for hit productions of Blood Brothers and Joseph and the Amazing Technicolour Dreamcoat, as well as recent West End shows such as Heathers: the Musical. Kenwright acquired the rights to It’s a Wonderful Life and approached McCartney – a fellow Liverpudlian – with the idea three years ago.

“Like many of these things, this all started with an email,” McCartney said. “Bill had asked if it was something I might be up for. Writing a musical is not something that had ever really appealed to me, but Bill and I met up with Lee Hall and had a chat and I found myself thinking this could be interesting and fun.”

‘Comedy, pathos and rare humanity’ … It’s a Wonderful Life. Photograph: Allstar/RKO/Sportsphoto

McCartney described It’s A Wonderful Life as “a universal story we can all relate to”. Capra’s film stars James Stewart as George Bailey, a selfless but embattled man contemplating suicide on Christmas Eve after he is framed for defrauding customers of the bank in his small American town. A guardian angel is dispatched from the heavens, and shows Bailey what the world would have been like without his lifetime of kind deeds, resulting in one of the most sentimental – and satisfying – endings in cinema.

The film was named the UK’s favourite Christmas film in two separate polls in 2018, by Radio Times and Odeon cinemas. Kenwright originally attempted to get the rights as a young man, but Capra turned him down in a handwritten letter.

Hall said It’s A Wonderful Life was his favourite film. “It has absolutely everything: comedy, pathos and a rare humanity which has touched generation after generation. Yet it just couldn’t be more relevant. To give it a life on the stage is an immense privilege, but to do it with Paul McCartney is off the scale. Paul’s wit, emotional honesty and melodic brilliance brings a whole new depth and breadth to the classic tale. I feel as if an angel must be looking after me.”

Kenwright, who worked with Hall to tour his 2007 play The Pitmen Painters, said working with McCartney was “a dream realised”, praising the musician’s “unique gift of melody and composition”.

‘It’s a Wonderful Life has absolutely everything’ … Lee Hall. Photograph: Linda Nylind/The Guardian

McCartney already has form in writing seasonal hits – Wonderful Christmastime reached No 6 in 1979 in the UK, and he recorded his version of The Christmas Song, popularised by Nat King Cole, in 2012 – but hasn’t commented on the songs being written for It’s a Wonderful Life.

Kenwright has heard demo versions, and said they “exceeded expectations … The songs take you somewhere you don’t expect to go. They sound simple – but it’s deceptive. That’s Paul’s genius.”

While he still tours, playing solo songs as well as tracks by the Beatles and Wings, McCartney refuses mere nostalgia, and has steadily released new music over the past decade. In 2012, he performed live with the remaining members of Nirvana; in 2014, he collaborated with rapper Kanye West, and that year he also wrote a song for the video game Destiny.

On his most recent album, Egypt Station, in 2018, he collaborated with contemporary pop songwriters and producers Greg Kurstin and Ryan Tedder. It reached No 1 in the US – his first chart topper in the country for 36 years – and was described in the Guardian as “an affirmation of an enduring talent, the work of an artist who has no need to try and be anything other than what he is”.

He has dabbled in musical film, first starring in jukebox musical films with the Beatles – A Hard Day’s Night, Help!, Magical Mystery Tour and the animated Yellow Submarine – which blended loose storylines with Beatles numbers.

In 1984, he wrote and starred as himself in Give My Regards to Broad Street, a film that paired new and old McCartney songs with a tale about a nefarious corporate takeover at a record company. It produced the hit single No More Lonely Nights, but the film itself was panned, with US critic Roger Ebert saying: “The usual thing is to see the movie and buy the soundtrack. With Give My Regards to Broad Street, I think you can safely skip the movie and proceed directly to the soundtrack.”