Clive Barker has always delighted in revealing the terrible darkness gathered just beyond the veil, ready and waiting for us to yield to the temptations of our inner ghosts and darkest desires. Shock and terror are short-lived emotions, however: you either recover or their cause kills you. It’s also possible that careers in horror are inherently time-limited. It’s one thing to make a splash with extraordinary early work, as Barker did with the seminal Books of Blood, novels such as The Damnation Game and Weaveworld, and the successful Hellraiser franchise, quite another to keep finding worthwhile blood to spill for decade after decade. Stephen King has managed it — and Peter Straub, too — but it’s rare.

The Scarlet Gospels is Barker’s first adult novel since 2001’s Coldheart Canyon (excluding the short and patchy metafiction Mister B Gone), and sees the return of two of his most compelling characters. Harry D’Amour is a private eye locked in endless weary battle with the dark, a man covered in tattoos protecting him from spirits and demons. D’Amour first appeared back in the Books of Blood and has popped up since, notably in the (possibly too) expansive Everville. In this new novel, he is pulled into climactic confrontation with Barker’s most iconic creation of all, the high priest of Hell, Pinhead – who has designs on an ultimate triumph.

Along the way we meet denizens of both Hell and various hells on Earth, the kind of characters at which Barker excels – lustrously damaged, polymorphously perverse individuals who might be broadly defined as “people you never want to meet”. After a contained first section in which D’Amour limps back to New York following a bad experience in New Orleans, the novel’s metaphysical remit expands when he declines an offer he shouldn’t have refused, prompting a harrowing rescue mission as he tries to save his blind medium friend Norma Paine from Pinhead’s clutches.

Pinhead has always possessed enormous melancholy grandeur, and Harry D’Amour is a significant noir presence. The story that unfolds between them is meticulously framed, endlessly inventive and spun with rollicking good humour. The devil is in the detail, naturally, and Barker’s unique imagination remains extraordinarily fecund. For someone so aware of the magical resonance of language, his style can seem curiously mannered and a little old-fashioned, but contrasts effectively with curdled sexual references thrown in with the cheerful elan of a man to whom it’s simply not that big a deal. Barker has not merely made friends with his shadow, you sense, but taken him home for the night.

This is far more than a wallow in Grand Guignol, however. Many have tried to emulate Barker’s confidence with the appalling, but their work often feels like a mere piling on of words, designed to shock: naughty children dabbling their hands in filth, in hopes of disturbing whichever authority figures they yearn to unsettle. Barker, one suspects, really doesn’t care. This is where he lives; these are the stories he has to tell; and he’s come out with newly sharpened knives. Most writers of horror fiction produce novels bedded in the “real” world, into which chaos intrudes. Barker starts from the other side. This is a tale of tattered misfits fighting for their souls in realms of dark wonder and tragic import, into which sparks of normality float – casting a strange, unsettling light on the reality most of us call home.

Perhaps because of his work in young adult fiction in recent years, in The Scarlet Gospels Barker applies himself to a more focused canvas, and his emotional palette is the richer for it, with striking moments of real tenderness. He’s in tight control of his prose, and there’s a lightness of touch that’s sometimes been missing in the past. In short, this reads like a novel by a man who’s glad to be back, and has plenty of sights to show us. It’s also, if you’ll excuse the pun, a damned good read.

It’s a strange old universe. Come see its dark side, if you dare. Bring a spare set of clothes.

• The 20th anniversary edition of Michael Marshall Smith’s first novel, Only Forward, is published by HarperCollins. To order The Scarlet Gospels for £15.19 (RRP £18.99) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.