If William Shakespeare wrote about his son’s death at all, he concealed it in the lines of his late sonnets and plays that reveal a depth of understanding about grief. Now the latest in a run of modern works to be inspired by the sad legacy of this short life is being heralded as one of the most eagerly awaited books of the spring.

The acclaimed author Maggie O’Farrell’s first historical novel, Hamnet, tackles the mystery surrounding the fate of Hamnet Shakespeare, the playwright’s only son. It follows the critical success of the 2018 film All Is True, which starred Kenneth Branagh as the Bard and Judi Dench as his wife, Anne Hathaway, and of Bush Moukarzel’s short play for a solo child actor, Hamnet, which premiered in Dublin two years ago.

“I have been fascinated by Hamnet ever since I first studied Hamlet at school,” O’Farrell has explained. “I was always baffled and saddened by how little mention he receives in biographies and literary criticism, so I decided to write a novel about him, to attempt to give him a voice and a presence.”

O’Farrell’s Hamnet, out at the end of March, takes an unconventional approach to literary history and has already been hailed as a critical hit, with the award-winning writer Kamila Shamsie declaring it: “Dazzling. Devastating”.

In the novel, O’Farrell, 48, a Costa award-winner in 2010 for her book The Hand That First Held Mine, follows young Hamnet, who is thought to have been named after a family friend Hamnet Sadler, as he searches for help for his gravely ill twin sister.

The agreed facts are that Hamnet was indeed twin brother to Shakespeare’s second daughter, Judith, and that he died at the age of 11. He was buried in the churchyard at Holy Trinity in Stratford-upon-Avon in the summer of 1596, one of the most productive years of his father’s early London career. In an era of frequent plague outbreaks, the cause of death was not always set down in parish registers, so the circumstances of the boy’s death are uncertain. It is also unclear whether Shakespeare attended the funeral.

“The twins had been baptised at Candlemas 1585,” said Greg Doran, artistic director of the Royal Shakespeare Company. “According to John Manningham’s diary, the first recorded performance of Twelfth Night took place exactly 17 years later on 2 February 1602. Judith no longer had her twin to celebrate their joint birthday. As a twin myself, I find that poignant. Her father can reunite his fictional twins, separated in a shipwreck, and for Viola and Sebastian the aching loss can be magically healed, but he can do nothing to heal his family’s grief revived every birthday.”

Scholars have pointed to the constant outbreaks of bubonic plague during this period and assumed that Hamnet contracted the disease. But in All Is True, the writer Ben Elton imagined that Hamnet might have drowned in a Stratford pool. His story began with Shakespeare’s return to his Warwickshire home in 1613 after a fire had burned down the Globe theatre. Shakespeare was to live only another three years, dying at the age of 52. Hathaway died in 1623 at 67.

David Mitchell as the Bard in Upstart Crow. Photograph: Colin Hutton/BBC

Hamnet’s death threw a shadow over Shakespeare’s final years. Elton’s latest comedy, the West End show The Upstart Crow, which stars David Mitchell in a reprise of his television sitcom role as the playwright, refers again to the family’s burden of loss.

“It’s hard to say what darkens Shakespeare’s plays as the century tips into the uncertain seas of the new century,” said Doran, “but they seem to topple into an abyss in the early Jacobean period. The threat of terrorism, the gunpowder plot, a sense of the country and the world loosing its moorings, division as the new king strives for union, all seem familiar, and are perhaps reflected in his plays.”

The darkest notes of plays such as The Winter’s Tale and King Lear, which handle family death, are sometimes ascribed to Shakespeare’s bereavement, but other experts go directly to lines from King John, where Constance, a grieving mother, says:

“Grief fills the room up of my absent child,

Lies in his bed, walks up and down with me,

Puts on his pretty looks, repeats his words,

Remembers me of all his gracious parts,

Stuffs out his vacant garments with his form.”

(King John, Act III, Scene 4)

An RSC production of The Winter’s Tale opens at Stratford on 28 March, and for Doran its more optimistic notes may also have biographical roots. “It is both tempting and beguiling, while probably deeply misleading, to read too much into the plays,” he said. “Is it significant that his first grandchild, Elizabeth, is born in 1608, and would have been two or three as her grandfather begins to write the late plays, with their sense of hope in the next generation? ‘Welcome hither as is the spring to the earth’ is how Leontes greets the daughter who has been restored to him.”

We do know, however, that in 1599 Shakespeare began to write the role considered the greatest of all time: the historical Danish prince Hamlet. O’Farrell cannot shake off the idea that the themes of this play are linked to his lost son. “His death at such a young age, and his connection to the subsequent play is a story I’ve been thinking about for a long time,” she has said.