When Hilary Mantel gave her BBC Reith lectures in 2017 on the subject of historical fiction, she was asked by one audience member if she was really writing ghost stories. She replied emphatically: “Yes,” almost before he had finished the question. Ghosts have peopled her fiction from her first novel, Every Day Is Mother’s Day, and her memoir, Giving Up the Ghost, made clear how permeable the boundary between the living and the dead can seem.

“St Augustine says the dead are invisible, they are not absent.” Mantel opened her first lecture with this quotation, and her historical novels are such towering achievements partly because she appears to inhabit the thoughts and sensory experience of her characters so fully – not so much as if she is channelling Thomas Cromwell, but as if she, disembodied, has projected herself back into his mind and drawn the reader with her; as if we are, in effect, ghosts at his shoulder, seeing through his eyes. This effect is particularly noticeable in The Mirror and the Light, the long-awaited third instalment of her trilogy about the blacksmith’s son from Putney who became Henry VIII’s right-hand man and the architect of much Tudor policy that reverberates down to the present.

It’s been eight years since the publication of Bring Up the Bodies, for which Mantel won a second Booker prize, but we find Cromwell exactly where we left him, in May 1536, at the moment the French executioner has struck off Anne Boleyn’s head with his sword. Cromwell is much taken with this sword: Toledo steel, incised with the words of a prayer. It is only later in the book that we learn the words on the blade: Speculum justiciae, ora pro nobis. Mirror of justice, pray for us. It’s one of many references to both mirrors and light stitched subtly throughout the book, though the title comes directly from a line Cromwell offers Henry:

“Your Majesty is the only prince. The mirror and the light of other kings.”

Henry repeats the phrase, as if cherishing it: the mirror and the light.

The question of whether Anne and the men executed for adultery with her received justice is one that haunts many of the characters in the novel, not least the king and Cromwell himself.

Henry says: “Did I do right?”

Right? The magnitude of the question checks him, like a hand on his arm.

Facebook Twitter Pinterest ‘We walk with him towards an outcome that we know but he does not’ … Mark Rylance as Thomas Cromwell in the BBC adaptation of Wolf Hall. Photograph: Giles Keyte/BBC

But the dead are not absent. In the Tower of London, Cromwell sees shades of the men whose deaths he contrived, and feels their company when he steps into his barge; at night, he hears footsteps in his bedroom and dreams of Anne’s execution as if it’s part of a series of altar paintings. The answer he gives Henry – “Go forward, sir. It’s the one direction God permits” – might be taken as his personal credo, but Cromwell’s path forward takes him along a knife’s edge. Anne’s death has not made his own position more secure – if anything, the reverse, as the imperial ambassador Chapuys gleefully reminds him: “For when all is said, you are a blacksmith’s son. Your whole life depends on the next beat of Henry’s heart, and your future on his smile or frown.”

The reference harks back to the vivid scene in Bring Up the Bodies when Henry is injured in a tournament and believed dead. Cromwell’s instinct is to flee, but in the event he is the only one who dares lay hands on the king and thump him back to life (his French servant Christophe claims he grabbed the king and bellowed into his face: “Breathe, you fucker, breathe!”). But Chapuys’s words also offer a grim dramatic irony, one instance of many throughout the novel.

Because we all know how the story ends. And this is where Mantel’s supreme artistry is most evident: she creates suspense and apprehension where none should exist. “You succeed not despite the fact that your reader knows what happens, but because of it,” she has said. The present-tense, Cromwell’s-eye narrative means we walk with him towards an outcome that we know but he does not, so that we are both with him and at a distance, helpless to warn him. When, dividing up the spoils of the dissolution, he picks out the monastery of Launde as a modest, pleasant place he might claim for his own retirement, we remember Francis Bryan’s prediction: “If the king turns on you, it will not be like when old Henry Guildford quit and went off to the country to prune his fruit trees and enjoy the birdsong. Remember how Wolsey fell.”

One reviewer complained of Wolf Hall that it had no plot, which strikes me as a wilfully obtuse failure of reading. These books are precision-engineered, and none more so than The Mirror and the Light. It may be less obviously dramatically focused than Bring Up the Bodies, which spanned less than a year and concentrated almost exclusively on events leading up to Anne’s death, but the plot here is shaped as meticulously as any thriller. Chekhov’s gun is there on every page: words spoken carelessly or in jest are later repeated in a court of law, their meaning twisted; gifts given in innocence are produced with new motives ascribed. The technical skill required to marshall the events of these four years between 1536 and 1540 – which include the dissolution of the monasteries, the northern uprising known as the Pilgrimage of Grace, and the political manoeuvring that resulted in Henry’s short-lived fourth marriage to Anna of Cleves – while rendering those events comprehensible and dramatic to contemporary readers, is breathtaking.

It’s impossible not to miss Anne Boleyn; the force of her ambition, and the savage verbal sparring that she enjoyed with Cromwell. Mary, Henry’s daughter by Katherine of Aragon, is a pale substitute, though she, too, attempts to pit her will against Cromwell. Mantel portrays her as a lonely, stubborn young woman, inviting pity but not sympathy, one moment giving in to emotional outbursts, the next desperately trying to assume the lofty dignity of a princess. Her half sister, Anne’s toddler daughter Elizabeth – now known as My Lady Bastard – is most often encountered furiously screaming. The only other woman who comes close to Anne’s vitality on the page is her sister-in-law, the waspish and conniving Jane Rochford, who allows Mantel’s foreshadowing to reach even beyond the limit of this book. “He thinks, you have destroyed one queen, is one enough?” Later, we meet flirty, teenage Katherine Howard, destined to become the second of Henry’s wives to be beheaded, along with Lady Rochford, who arranged her adulterous trysts.

Despite the absence of Anne’s glittering wit, the novel is shot through with wry comedy. There is a glorious scene in which the innocent, literal-minded new queen, Jane Seymour, is cornered by her family and Cromwell, all anxious to inquire, delicately, how the wedding night went. Jane earnestly expresses some consternation: “My difficulty is, he wants me to do some very strange things. Things I never imagined a wife had to do.”

There is nothing sentimental in Cromwell’s end, only the most devastating humanity, leaving the reader with stopped breath and a sense of amazement that the real world is continuing outside

The frantic glances exchanged over her head are so visually perfect – “Edward looks desperate. Tom begs, “Master Secretary?” – that the scene makes you long for the next instalment of the TV adaptation. Edward manages to ask: “… does it conduce to getting a child?” “I wouldn’t have thought so,” Jane says. It is left to Cromwell to coax from the queen what exactly the king asked of her in the bedchamber – which is, of course, the punchline to the scene.

But Jane does not last long, and the quest to find Henry’s next wife provides Cromwell – promoted first to baron and then to earl – with an opportunity to continue his project of remaking England. The nation’s founding myths have permeated all three books, and Cromwell is uniquely placed to select how much of that is useful, and how much can be erased, like the entwined initials of Henry and Anne that must now be painted over in every royal palace. But ultimately his fate is to discover that the past – both his own and his country’s – insists on reappearing.

“Can you make a new England? You can write a new story. You can write new texts and destroy the old ones, set the torn leaves of Duns Scotus sailing about the quadrangles, and place the gospels in every church. You can write on England, but what was written before keeps showing through…”

Cromwell’s desire to forge a political alliance with the German Protestant states through a marriage with Anna of Cleves, thereby creating an alternative power bloc in Europe, proves his downfall. Henry takes a dislike to his bride, though Mantel challenges the received idea that it was because Anna was unattractive; in her version, it is the young woman’s unguarded reaction to Henry’s bloated, ageing appearance that wounds the king’s pride and turns him against her. Inevitably, blame falls on Cromwell.

Thomas Cromwell review – new biography of the hero of Wolf Hall Read more

We have always known where it would end. From the first page of Wolf Hall, when we saw through the eyes of 15-year-old Thomas as he lay bleeding on the Putney cobbles, watching his father’s boot as Walter yelled “So now get up”, we were always going to share that perspective again as his head rested on the executioner’s block. Anyone who has read Mantel’s masterly novel of the French Revolution, A Place of Greater Safety, will know that she can lead her characters to the scaffold with such fine and telling detail that the reader experiences a visceral reaction to the terror of death. She did it again in Bring Up the Bodies with the execution of Anne Boleyn; the small detail of Anne, blindfold, reaching up to adjust her cap, and Cromwell in the wings, willing her to get her hand out of the way so the headsman can make a clean strike. There is nothing sentimental in Cromwell’s end, only the most devastating humanity, leaving the reader with stopped breath and a sense of amazement, after closing the book, that the real world is continuing outside.

It feels redundant to state that The Mirror and the Light is a masterpiece. With this trilogy, Mantel has redefined what the historical novel is capable of; she has given it muscle and sinew, enlarged its scope, and created a prose style that is lyrical and colloquial, at once faithful to its time and entirely recognisable to us. Taken together, her Cromwell novels are, for my money, the greatest English novels of this century. Someone give the Booker Prize judges the rest of the year off.

• The Mirror and the Light by Hilary Mantel is published by Fourth Estate (£25). To order a copy go to guardianbookshop.com. Free UK p&p over £15