Again and again in M John Harrison's extraordinary novels, those who are tasked with interpretation find themselves baffled, frustrated or worse. Detectives, theorists, scientists and code-breakers are all left intellectually concussed by their attempts to construe complexity. Even the machines of unimaginable hermeneutic power that inhabit his science fiction are unable to process the universe successfully. Some surplus of dark matter always exceeds their comprehension, some aspect of pattern persists that will not be translated. Signal is ultimately inextricable from noise, and message from clutter.

This resistance to conclusion distinguishes Harrison's legendary science fiction, and is also a hallmark of his "real-world" novel, Climbers. First published in 1989, and at last brought back into print this month by Gollancz, it is surely the best novel about rock-climbing ever written – though such a description drastically limits its achievement. In Climbers, there is no ascent to enlightenment. The book never, in the language of the sport, tops out. Mike, the rootless main character, spends a year climbing and ends it no wiser than before. Near the novel's close, he solos a route on a Welsh sea-cliff that he had ascended the previous summer. He rests on a sunny ledge 150 feet up, thinking back over the past 12 months. But his only revelation concerns the persistence of confusion: "I realised I didn't know any more than I had the last time I sat there. I didn't know anything about anything."

Hindsight is no clarifier in Harrison's world, overview no advantage and analysis no virtue. His great – and modern – subjects are indecision, bewilderment, estrangement, not-knowing, and the "empty space" between people: these are all experiences that cannot be written about conclusively. The content of his prose defies extraction, conversion and consumption: in this way, as in others, his politics are grained deep into his style.

Harrison is best known as one of the restless fathers of modern SF, but to my mind he is among the most brilliant novelists writing today, with regard to whom the question of genre is an irrelevance. To read his work is to encounter fiction doing what fiction must: carrying out the kinds of thinking and expression that would be possible in no other form. I pass through his novels feeling a mixture of wonder, calmness and disturbance; I end them brain-jarred and unsettled. Metaphysical echoes persist for days afterwards. It feels as if I have had a strabismus induced, causing illusions that slowly resolve into insights.

His prose is often described as bleak, but in fact it practises a parity of gaze. He offers lucidity without pity, but without rancour either. Although the fierce ease with which capitalism husks humanity is one of his main concerns, though debris (emotional and material) is one of his chief preoccupations, and though rupture is the texture that most attracts his eye, his vision is not devoid of either tenderness or hope.

For despite his interest in loss and harm, Harrison is also fascinated by the gifts that life gives to even the most dispossessed of people. His books gleam with the lustre of found beauty and brief happiness: moments of being that exceed the claims of market forces. This aspect of his work is, I think, what led Michael Moorcock memorably to christen him "an anarchist aesthete". There are no candy cane epiphanies or take-home morals – just occasional light-flares from unexpected sources and surges of sensation that might make existence worth enduring.

There is little point trying to summarise the plot of Climbers, for it does not really deal in plot. What would one say? That a man called Mike in flight from a failed marriage moves to Yorkshire and becomes involved with the climbing scene. That the book follows him and his friends as they climb, drink, laugh, fuck, and fuck up. Such a summary falls foolishly short of the thing itself. What might be added? That one need not be a climber to read this novel, for one of its implications is that we are all climbers of a kind. That it borders on the memoir ("Mike" shares not only a first name but several biographical details with Harrison). That it is a novel that surely knows more than its narrator, and perhaps more than its author, for it does its thinking at the levels of form and style. And that it is a remarkable evocation of the messed-up and broken-down landscape of contemporary England.

The England about which Harrison writes here, though, is far from the bosomy downland dreams of Batsford. This is a late-Thatcherite England – mostly northern – of contamination, industry, unemployment and discard. To evoke it, Harrison creates a language in which the natural and the artificial are fully fungible. Dribbles of ice on winter rock are "shiny as solidified superglue". Hedge mustard is "stark as a tangle of barbed-wire". Everywhere, the toxic and the beautiful twine or swap with one another: in a public lavatory in spring, the "cubicles [are] full of the smell of hawthorn, strange and lulling in a place like that, which overpowered even the reek of piss".

This is landscape writing, but not as we know it: descriptions so scoured of the delusions of the pastoral and the grandiosities of the sublime that the places they record seems surreal for being true. Harrison does this repeatedly, stripping back conventions to leave places skewed and startling. He writes at one point of how months of drought cause the water levels in the northern reservoirs to drop, "revealing strange fossil beaches, submerged cliffs and channels, a monolithic architecture of tunnels and ramps". It could stand as an image of his own method – draining away the familiar to reveal its weird sub‑structures.

Harrison's Britain is not denatured. Life persists; the green fuse still burns. But "nature" is a compromised category, which expresses itself in ecologies of neglect and corruption, as in the London basement where "a mysterious vitality had caused its walls of greyish brick to grow damp moss, and in one place small clumps of willow herb and bright yellow ragwort". The climbers themselves are versions of these weedy species, clinging to their gritstone walls, finding their niches in quarries where the west wind blows in "sulphur from the badly managed factories in Stockport and Stalybridge", stoking their "vitality" with beer and cigarettes.

The climbing they do is impure, tending to the tawdry. It lacks the cleanliness of winter mountaineering, or the epic scale of big-range expeditions. It is mucky, thrutchy stuff that happens from litter-strewn crag-foot terraces, within eyeshot of cities and earshot of motorways. Nevertheless, it has its honour codes, its idiolect, and – because it is nearly as easy to die from 20 feet up as from 2,000 – it offers its practitioners a combination of fear and excitement that is "indistinguishable from joy".

Save for that fearful joy, little in Climbers signifies in the way it should. Seemingly crucial events flit past in a few sentences, barely registered. The many deaths and injuries that occur are shocking for the distracted scarcity of their narration, like the boy from Lancaster who "panicked above the crux of quite an easy route called Touch of Class and fell with a groan of fear backwards on to zinc grey boulders the size of commercial refrigerators". What a sentence that is – and how hard I will find it to forget it. Even as Mike recalls someone's desperate accident, his climber's mind is still grading the climb, and implicitly disdaining the boy for failing on it. The "groan of fear" is chilling, but chillier still is the image of the boulders as "commercial refrigerators", which brings a white-goods mundanity to the event, but also suggests the unyielding edges and corners on to which the boy plummets. Shades here of the Smiths, a band not mentioned in the novel but whose music haunts it: "Young bones groan / And the rocks below say / 'Throw your skinny body down, son'".

Climbers is usually seen as an outlier in Harrison's oeuvre – a rare real-world expedition – but its kinships with his other work, especially his dystopian The Committed Men (1971), or the dazzling Kefahuchi Tract trilogy of Light (2002), Nova Swing (2006) and Empty Space (2012), are unmistakable. But then to Harrison, all life is alien: the inhabitants of a motorway service station near Penrith or an inner-city Sheffield cafe are as compellingly bizarre as anywhere in The Halo or Radio Bay. The climbers have their SF counterparts in the "shadow operators" of the Kefahuchi novels, the snickering figures who haunt the trilogy's periphery, "algorithms with a life of their own … full of crime and beauty and inexplicable motives". The hailstones that fall like "bone dice" on the Yorkshire crags will later return as The Shrander's stolen goods in Light. Even the free indirect style in which Climbers is told, slipping restlessly from consciousness to consciousness, anticipates the technology that permits Light's pilot Seria Mau to inhabit her "K-ship" as a dispersed extension of her own mind.

Rock-climbing is, at its exhilarating best, a free indirect form of motion, in which the climber becomes – as Mike puts it – "the idea or intuition that sat cleverly at the centre of [the climb], directing it". I have never been nearly a good enough climber to enter this state myself, though I have watched it happen to others. The climber achieves a faultless fluency in which instinct absorbs the role of conscious choice, holds leap to hand, and static rock and shifting body appear to flow together. Harrison seems to write his novels in some chronic version of this state. I have read nearly 2000 pages of his work, and cannot recall a single mis-step or over-reach – only a sustained and mobile grace.

• Robert Macfarlane is chair of this year's Man Booker prize judges, and the author of The Old Ways.