Last year was a significant one for China Miéville. The City and the City won him the Arthur C Clarke award, science fiction's most significant prize, for an unprecedented third time, and also brought mainstream critical applause. Kraken was published and his new novel, Embassytown, was in preparation. He marked the year with an arm-spanning tattoo of a "skulltopus", a grinning skull swathed in vibrant tentacles, an image developed as a homage to the different traditions of the weird and fantastic from which his imagination springs.

Miéville has always worn his influences on his sleeve – Lovecraft, Peake, classic and new wave SF, fantasy, comics and the Dungeons and Dragons role-playing games he played as a kid – but from the start his books combined this love of genre, geeky in its enthusiasm and scholarly in its depth, with an ambitious literary sensibility. Embassytown, published this month, takes that ambition to a new level. An investigation into culture shock and the links between language and thought, it's the story of a backwater planet colonised by humans whose attempts to communicate with the alien "Hosts", who have no concept of lying, go very badly wrong. But while the metaphysical implications of creatures for whom there is no gap between a word and its referent reach back to postwar linguistic philosophy, Wittgenstein and beyond, the original idea was of a dual-voiced alien, and it came to Miéville when he was 11. "I have incredible fidelity to my own obsessions, which is a dignified way of saying arrested development," he says. "I recently found the exercise book in which I'd written an early draft of what became Embassytown a quarter of a century later. It's amazing how much these things don't change."

Miéville was born in Norwich in 1972 but moved to the capital as a small child after his parents separated. His first memories are of London, which dominates his work: "I feel like London inhabited me from quite a young age as much as vice versa." He still lives in the same patch of north London where he grew up with his mother, a teacher, and his younger sister. His father died when Miéville was 19; after the separation he met him only a handful of times, leaving "odd and discombobulated" memories behind.

Miéville's passions crystallised early on: "Ever since I was two, I've loved octopuses, monsters, abandoned buildings . . . One gets asked, if you're into the sort of thing I'm into, how did you get into it, and my response is always: how did you get out of it? You look at a class of six-year-olds, they're all reading about witches and aliens and spaceships and magic spells." He wrote stories and poems as a child, and remembers "self-consciously thinking 'ooh, maybe this is what I could do' when I was about 13". Later, "I realised how lucky you would have to be. I never had an unthinking faith that this was what would happen."

When his mother moved out of London, Miéville went on a scholarship to board at the public school Oakham, where he spent a couple of "very unhappy years". After a gap year in Egypt and Zimbabwe, he took up a place at Cambridge to read English, but finding the teaching "fairly hermetic and abstracted" swiftly switched to anthropology. It was the point at which, intellectually as well as politically, Miéville came into his own. As a youngster, he'd been involved in CND and anti-apartheid campaigns; now he formalised his leftwing politics into an overarching Marxist philosophy. A masters in international law at LSE followed, along with a year at Harvard.

Miéville is often asked where his revolutionary politics and his fantastical world-building meet, but is wary of making too strong a connection between the two. "I'm not interested in fantasy or SF as utopian blueprints, that's a disastrous idea. There's some kind of link in terms of alterity . . . If you think about the surrealists, the estrangement they were trying to create was a political act. There's some shared soup somewhere in my head from which these two things are ladling."

His first novel, King Rat, published in 1998, was a twisted version of the Pied Piper story set in London's clubland, with drum'n'bass coursing through its prose. Miéville now feels "strangely affectionately embarrassed" for its cocky poise. The novel reads like a manifesto for his obsessions: London, both everyday and arcane; a radical political sensibility; and a determination to resist the standard tropes of fantasy whereby quests are followed, chosen ones fulfil their destiny and evil is vanquished. Iain Sinclair, one of Miéville's heroes, found in it a "genuine contribution to London's subterranean mythology"; his mother, reading this coming-of-age tale in which father figures are defenestrated or turn out to be giant rats, remarked "Search for the lost father much?"

He had been mapping out the alternate universe of Bas-Lag for 10 years before Perdido Street Station, a 900-page slab of baroque fantasy, was published in 2000. It's an extraordinary, sprawling world, powered by magic and steampunk technology, populated by humans, cactus-people, insectoid, amphibian and avian races, dripping with myths and monsters and menaced by repressive regimes. Michael Moorcock today compares it to Gormenghast. "What distinguishes China's invented world is the complexity and detail he gives it – and the believability of its characters, whether they are human or giant bugs."

At its centre is New Crobuzon, "a coagulum of all the cities I love in reality but also very much in fiction. London looms larger than any other city, but the literary and refractory London as much as the real London . . . Some people rigorously work through almost a Darwinian attempt to world-create in fantasy. I wanted to back-project rigour but start with a ragbag." Two more fat tomes followed: The Scar, a picaresque maritime adventure in which the city at the heart of the book is a floating community of ships lashed together by pirates; and Iron Council, a politically charged western in which a train hijacked by revolutionaries strikes out into the unknown.

While writing the Bas-Lag novels, Miéville had also been working on a PhD in philosophy of law, and continuing his grassroots activism. In 2001 he stood as a Socialist Alliance candidate, gaining just over 1% of the vote. "After the first New Labour government it had become very clear quite how hard they were pulling Labour to the right. It was very important to us that an alternative agenda was put." Ten years on, he says, "I find the political conjuncture toxic, vile and really upsetting. But I don't think there's any contradiction between being politically optimistic and thinking we live in a really bad moment. Quite the opposite."

The Bas-Lag books put Miéville at the forefront of a group of writers who blended science fiction and fantasy elements with horror and pulp into what was enthusiastically labelled the New Weird: dark, politically aware urban visions that explicitly rejected the consolatory, escapist strain established by Tolkien. For many, fantasy is typified by The Lord of the Rings; Miéville worked up a righteous fury against Tolkien's "cod-Wagnerian pomposity, his small-minded and reactionary love for hierarchical status-quos", calling him "the wen on the arse of fantasy literature" and setting out to "lance the boil". (Recently he's softened his position, not least because he "started to be asked at conventions to 'do the Tolkien thing', so I tried to shut up about it".)

Miéville became "exemplary of a moment", he admits now. "For people who don't know the field, I get used as shorthand for interesting stuff going on. You'd be kidding yourself if you thought it was all down to your innate wonderfulness." Yet from the beginning of his career the literary mainstream also sat up and took notice, even if it was sometimes only to discount him. For the 2003 Best of Young British Novelists issue Granta editor Ian Jack namechecked Miéville as "an extraordinary writer of dark fantasy" but stopped short of including him on the list.

Since Miéville began writing, the snobbery and divisions that plague discussions of genre have begun to weaken, as have the lines between them. "Although genres can be fantastically insular, there's a lot of excitement both from within and without when things do bleed. It happened with cyberpunk, and in the early 70s with New Worlds. We're at a fairly good moment, where there is a lot of borrowing and openmindedness." But although Miéville is frustrated by "the endlessly arse-achingly expressed complaint from genre that no one takes us seriously", he admits that slights from the mainstream continue. "'When are you going to start writing proper literature, reading proper literature' . . . When did the LRB last do an article on the amazing cutting-edge stuff going on in SF?"

One recent development in the debate around genres is an increasing discussion of "litfic" as a genre in itself. As M John Harrison, another of Miéville's literary heroes, recently wrote in his blog, "The sooner literary fiction recognises and accepts its generic identity, the sooner it can get help." Miéville heartily concurs: "I love genres; I think they are fascinating. My issue with litfic is not that it is a genre but that (a) it doesn't think it is and (b) it thinks it's ipso facto better than all the ones that are genres. Literary fiction of that ilk – insular, socially and psychologically hermetic, neurotically backslapping and self-congratulatory about a certain milieu, disaggregated from any estrangement or rubbing of aesthetics against the grain – is in poor shape." Miéville identifies Ian McEwan's Saturday, set around the 2003 demonstration against the Iraq war, as a "paradigmatic moment in the social crisis of litfic".

"In the early 2000s there was this incredible efflorescence of anger and excitement . . . It seemed to me that Saturday quite bolshily said, 'OK, you accuse us of a neurotic obsession with insularity and a certain milieu. I'm going to take the most extraordinary political event that has happened in Britain for however many years and I am going to doggedly interiorise it and depoliticise it with a certain type of limpid prose . . . It was a combative novel that met that sense of there being a crisis and de-crisised it through its absolute fidelity to a set of generic tropes."

Following a children's novel, Un Lun Dun, based in a fantastic alternative London, 2010's The City and the City, an existential murder mystery set across two opposing eastern European cities that occupy the same physical space, played with a new generic tradition: crime. "The funny thing is that for my least fantastic book, it started out of a very generic idea: a city that was inhabited by two different species, one a group of giants who were about three times the size of everyone else. You would have to have this concatenation of completely different buildings within the same city. That got me thinking about the political ramifications of two completely different communities living together. Slowly the fantastic started to bleed out, and the sociopolitical remained."

The novel prompted comparisons with Kafka and Philip K Dick for its exploration of arbitrary authority and individual disorientation, and has been read as an allegory of divided cities such as Jerusalem and Berlin as well as the quotidian willed blindness of modern life. Margaret Atwood calls the book "an intricately detailed metaphor for how we live today – ignoring what is right there in front of us but 'invisible' because we choose not to see it".

The book had been conceived as a crime novel partly as a gift for his mother, a fan of detective fiction. Miéville wrote the first draft through her long illness, first with breast cancer and then with leukaemia, a rare side effect of the chemotherapy used to treat her cancer. Her death in 2007, at the age of 58, hit him very hard.

If The City and the City marked a new direction, with a sparer prose and a more sombre tone, Kraken, published shortly after, "felt like the end of something". Miéville describes the book, a riotous mixture of London lore, messianic cults and pop-cultural in-jokes, as "an attempt to channel a sort of hopefully enjoyable ill-disciplined exuberance that I felt I had been moving away from". The book opens with the disappearance of a giant squid from the Natural History Museum. "There really is a preserved giant squid there. When I heard they had it I completely lost my shit, as a cephalopod-fan. It felt to me like a bottled myth in this room. It was just so affecting." It started with the squid, but soon "felt very much like a homage to everything I could think of. It's probably the most whimsical book I would write."

Embassytown is a much cleaner, more streamlined beast: Miéville knew he wanted to create a science-fictional universe this time, to carry the ideas about linguistics. He has also moved towards building up a sense of culture shock through withholding information rather than lathering on baroque descriptions. "One of the things I like about SF is not knowing what's going on. Nothing will ever breach my teratophilia, and I don't want to seem to be moving away from the monsters, but it's quite deliberate that in this book the descriptions of the aliens are very nebulous. It is about going into the words themselves, given that the whole book is about language and signification."

As well as being "neurotically about language", throwing in plenty of jokes about academics and linguistics, Embassytown is a sincere homage to its SF forebears. Miéville insists that "I would never disavow my generic tradition. Occasionally people say, 'but you're not really science fiction, you're escaping the genre'. Not really! I know it's meant nicely, but I would much rather operate as a conduit than an outlier." For Miéville, as for fans and critics in the SF field, genre is where the pulse of literature – the ideas, the excitement – is to be found. "The project of realism, the very name, shows it to be not merely hubristic, it's absurd, it's preposterous. Which bit? Which bit are you being realistic about?" Increasingly, Miéville is a locus of critical hopes, and Ursula K Le Guin is quietly confident: "When he wins the Booker, the whole silly hierarchy will collapse, and literature will be much the better for it."