Any genre needs its contrarians, says the writer M John Harrison. "It needs constantly reminding that it isn't the centre of the world." Over the last four decades, Harrison has carved out a stellar career on the margins of a fair number of literary forms. He caught science fiction's New Wave at the end of the 1960s, working with Michael Moorcock, JG Ballard and Brian Aldiss to push the genre towards the avant-garde. He subverted fantasy with the unstable universe of his Viriconium novels, won the Boardman Tasker prize for mountain literature in 1989 with Climbers, has built a reputation as a fine literary stylist with surreal novels and a host of short stories, and has set the tenor for much of modern British science fiction.

For many in the science fiction community, his 2002 novel Light is his masterpiece – Iain Banks hailed it as "a novel of full-spectrum literary dominance … a work of – and about – the highest order". Written in three strands, it braids together the tragedy of a late 20th-century serial-killer physicist with the 25th-century stories of one deadly pilot plugged into an alien rocket ship, and another who immerses himself in a virtual world based on 20th-century noir fiction. Lurking at its centre is a mysterious rip in spacetime called the Kefahuchi Tract, a naked singularity which "streams across half the sky, trailing its vast invisible plumes of dark matter". Harrison brings the trilogy to a close with Empty Space, picking up characters who seemed at first to be incidental to Light and 2006's Nova Swing, and putting them centre stage in a drama that moves between a downbeat, post-crash 21st century – which Harrison says was "gifted" to him by the 2007 financial crisis – and a 25th century mired in narcissism and meaningless conflict.

Buried near the core of the trilogy is an idea that would shock the scientists currently pursuing the Higgs boson: that any physics will do. In this world, alien races describe the universe in an entirely different way, using concepts that are totally incompatible with our own, but which still work. Harrison gives a mischievous laugh. "I enjoy being told off about that by physicists – that's a science fiction writer's proper place."

If Ray Bradbury wrote to forestall a future, Harrison says he writes to "forestall a present"; the militarised capitalism, environmental destruction and short-sighted self-obsession of his imaginary universe are only "a description of the world we live in". It's a brutal, soulless world where human beings see the unknowable wonder of the Tract only as a chance to make some money; people visit the "chop shop" to genetically modify their bodies to look like Marilyn Monroe or grotesquely over-muscled fighters; and the distant war against the aliens is "your war, to be accessed however it fitted best into your busy schedule". According to Harrison, the idea for the first book came after he got drunk with Iain Banks, who told him his trouble was that he didn't have enough fun.

"The next day I thought, 'Right, you sod, we'll see who can have fun,' and I just began." Light arrived in a steady stream, Harrison says, with the whole trilogy taking shape as he wrote it. "There are times when you know what's feeding the monkey is back here and down," he gestures past his pony tail down the back of his neck, "where there's a person who's probably you – but there's no proof of that – a person who knows more than you, who's got a very black sense of humour and a real sense of contempt for you, and you want him to speak."

Harrison loves to write something and think: "'Did I do that?' … and for a couple of days you have this real sense that you're haunted by another you." He doesn't apologise for the darkness in his work: for the sex, the violence, the damaged women who run through it. He describes himself as an "equal opportunities misanthropist", saying he's "skewered many more men than women, and possibly more donkeys than both".

Harrison relies on found material, "dialogue you've heard on the bus", to ground the far-away worlds he creates. A novel, or a short story, will begin, he says, when four or five things come together and "you don't know why". People talk about world-building "as if you build a universe brick by brick in front of the reader. You don't do that. You rely on accidents." He traces this patchwork method of writing, with material moving from notebook to short story, from short story to novel and back again, to reading The Waste Land when he was 14, an experience he says was like "being rolled over". Not only was he enchanted with the atmosphere of ruin that fills his own work, but he was also gripped by the way Eliot makes "a whole thing out of fragments and at the same time demonstrates that it is still fragmentary".

Harrison was born to an engineering family in 1945, and it was assumed he'd be an engineer in turn. But after his father's death, the teenager found himself "bored, alienated, resentful and entrapped", playing truant from Rugby's Dunsmore school. "This was the 1950s," he says, "everything exciting was going on outside the world defined by parents and school." An English teacher introduced him to George Bernard Shaw and within a month he was "hooked on polemic". Not that he stopped playing truant. "Back then, writing was just another way you could bunk off."

After leaving school at 18 he found himself "more out of employment than in", because all he wanted to do was write. "Writing's like gambling," he says. "Unpredictable and sporadic successes make you more addicted, not less." His number came up when Michael Moorcock, the editor of the pioneering SF magazine New Worlds, walked into his London bedsit. He began to write reviews and short fiction for New Worlds, and by 1968 he was appointed books editor. "They liked that I was perpetually angry," he says.

Working at New Worlds was "the only game in town" if you wanted to write imaginative fiction, according to Harrison. "It was like being able to swim in the stuff. You were able to read JG Ballard's latest short story as a manuscript, and that blew the eyeballs out of your head in 1969, 1970." He would go home, read Thomas Pynchon or Samuel Beckett and write into the early hours. "My head exploded for about five years." These detonations produced two novels in 1971 – a post-apocalyptic road trip, The Committed Men, and The Pastel City, the first in the Viriconium series set in a toxic, decaying world living off the technological scraps of previous cultures – an idea to which he returned on a galactic scale with the Kefahuchi Tract. But by the mid-1970s, after reading "quite a lot of rather poor science fiction novels", he began to feel a "real sense of horror, eventually, that this was not useful fiction. This is fiction that is supposed to be imaginative." It was time to go off and do his own thing.

Harrison's own thing turned out to be climbing, which he took up despite falling off the wall during a taster session in a north London sports centre. "I really wanted to investigate that moment in terms of my personality," he says. "Why do you want to put yourself in a situation where the one thing you fear the most is likely to happen to you, why do you keep putting yourself in that situation until it is very, very likely to happen?" He had spent 10 years writing fantasy, and at 30, he thought that if he spent any more of his life "sitting in front of a desk making things up", he would "lose touch with the actual physicality of the world".

He spent 20 years climbing, not in the beauty of the mountains, but in the brownfield belts around northern cities. "I've never been to the Himalayas," Harrison says, "and I'm not really interested in them. I'm more interested in a dirty old quarry in Lancashire, and by god, they can be dirty." This physical pursuit found its way into prose with a series of "somewhat more mainstream" short stories and his 1989 novel Climbers, which follows a man who falls in with a group of rock climbers. By 1992, The Course of the Heart found Harrison invading realism with the fantastical, before returning to SF with Signs of Life in 1996.

He sees all his work as part of a continuum, from the straightforward realism of Climbers to the "argy-bargy" of the Kefahuchi Tract. Dividing literature into genres is limiting", he argues, "a marketing device that got out of hand, and leaked into the audience". His work may feature SF's faster-than-light travel and sentient tattoos, but he rejects its appetite for heroes, which he considers to be "pernicious … a lie about the way the universe is constructed". Whether full of alien life or not, the universe is always ripe for human colonisation, he explains, "which seemed to me the kind of thing you'd want to write a space opera against."

"A good ground rule for writing in any genre is: start with a form, then undermine its confidence in itself," he says. "Ask what it's afraid of, what it's trying to hide – then write that." For Harrison, the most satisfying writers are "at odds with their cultural context. They're trying to fit in and failing, or they're trying to remove themselves and failing. The attempt to resolve the conflict is an angle – a frame or a context – in itself."

Harrison is often called a "writer's writer", a compliment that can cut both ways. How does he feel about this? In reply he describes the "practice crag" found in almost every Peak District town or village. "It may not be much higher than this room," he says, "but every single way of getting to the top of it will have been worked out over 50 or 60 years." At the same time, there will always be "some last great problem that nobody's solved. The guy who will solve it may not be the best climber in Britain, but the best climber in Britain will turn up one day in the summer to watch the local guy who can do it. And I always wanted to be that local guy, as a writer. To be that technical, that familiar with a certain locality, and within those terms, able."