Proof for me of writer/director Kenneth Lonergan’s much-vaunted cinematic genius has been a long time coming. Having co-written the Sopranos-lite comedy Analyze This (1999), the acclaimed playwright made his directorial feature debut with You Can Count on Me (2000), a Sundance festival favourite that earned rave reviews and Oscar nominations. But Lonergan’s second feature, Margaret, shot in 2005, became mired in post-production hell, delaying its release until 2011 when it crawled into cinemas, loved by critics but largely unnoticed by audiences. Now, after another lengthy hiatus, Lonergan is back with Manchester By the Sea, the story of a broken man returning to the scene of past traumas. Striking a delicate balance between the sharp focus and sprawling scope of his first two films, this heartbreaking third feature finally confirms Lonergan as an auteur of genuine merit rather than just exasperating promise.

Affleck is a silent scream waiting to explode, a swelling ocean of pain and rage beneath a surface of calm

Originally developed as a vehicle for producer Matt Damon (who was variously set to direct and/or star), Manchester By the Sea casts Casey Affleck as Lee Chandler, a reclusive janitor eking out a meagre existence in the Boston suburb of Quincy. When his brother, Joe, collapses, Lee is called to his former home town of Manchester, Massachusetts, where raised eyebrows and whispers of local infamy (“that’s the Lee Chandler?”) haunt this hollow-eyed figure. Finding himself unexpectedly in charge of his seemingly selfish nephew, Patrick (Lucas Hedges), Lee wrestles with the ghosts of his past while attempting to deal with an unforeseen future for which no one believes him to be prepared – least of all himself.

Emptiness is hard to portray, too often represented by blank stares and vacant gazes. Yet Affleck, who seemed so pitifully vulnerable in The Assassination of Jesse James by the Coward Robert Ford, gets the depth of Lee’s isolation just so. A montage of seemingly inconsequential handyman scenes brilliantly define his almost sociopathic withdrawal from the world, a hint of the cold brutality of The Killer Inside Me lurking at the edges of his deadpan glare.

Pointedly ignoring any displays of affection, Lee is as flat and frosty as the snowy scenery, crisply captured by cinematographer Jody Lee Lipes. His only emotional engagements are outbursts of anger: swearing at a tenant whose leaky plumbing he has laboured to fix, or punching customers at a bar after an evening of drinking alone. There’s desperation in every movement Affleck makes, from the defensive hunching of his shoulders to the almost imperceptible pursing of his lips. He is a silent scream waiting to explode, a swelling ocean of pain and rage beneath a surface of calm.

What a contrast this presents to the flashback scenes of Lee’s formerly garrulous life, which weave in and out of the narrative – intoxicatedly open, foolishly affectionate, oozing boozy bonhomie to the point of boorishness. As a husband he’s loving, if insufferable; as a father and uncle, he’s irresponsible yet devoted. Gradually, the threads of the dual timelines converge, juxtaposing awful revelations against scenes of recrimination and perpetually open-ended resolution.

Around this fractured figure swarms a vibrant supporting cast: Kyle Chandler as the brother whose head is strong but whose heart is weak; Lucas Hedges, inhabiting the netherworld between adolescence and adulthood previously explored in Lonergan’s 2002 stage play This Is Our Youth; and Gretchen Mol as Patrick’s mother, Elise, a shrill alcoholic from whom Lee’s family have long been estranged. Most importantly, there is Randi (Michelle Williams), the mother of Lee’s beloved children, separation from whom seems to have driven a dagger of icy self-loathing into his heart.

As always, Lonergan demonstrates a keen eye for the mundane rituals that quietly accentuate the anguish of his characters. Bereavement (a recurrent feature of his films) is somehow amplified by the dreary bureaucracy of form-filling, while an absurd tussle with the wheels of an ambulance gurney provides a moment of almost unspeakable tragicomedy.

‘Somewhat sidelined’: Michelle Williams with Casey Affleck in Manchester By the Sea. Photograph: Allstar/ StudioCanal

There are a few false notes. Albinoni’s Adagio has been used in everything from Rollerball to Gallipoli (not to mention the BBC sitcom Butterflies), and here it brings corny cultural clutter to an otherwise audacious central sequence. A shame too that, despite the promises of the poster image, Michelle Williams is somewhat sidelined, diminishing the complexity of Randi’s emotional turmoil as we remain tightly focused on Lee. In a rare turn of events, I found myself wanting the film to be longer, to give her character more space to breathe.

These are minor quibbles. As Margaret demonstrated, Lonergan’s films have always been more like works-in-progress than perfectly finished artefacts. For all its moments of discord, Manchester By the Sea sings its lonely song with tremendous heart and soul. No wonder it left me wanting more.