In 2004, while waiting for my novel about Henry James, Author, Author, to come out, I occupied myself by writing the introduction to a Penguin Classics edition of HG Wells's novel Kipps. In April I made this note in my very occasional diary:

Researching Kipps I came across in Wells's Experiment in Autobiography an interesting story of the ménage of Mr and Mrs Hubert Bland at Well Hall, Eltham. He was a Fabian, a philanderer who converted to Catholicism, she was E Nesbit. Possible material for a novel like Author, Author here. Sex, politics, children's literature . . . How much has it been worked over?

Little did I know, but it was probably being worked over at that very moment by AS Byatt, who five years later would publish her novel The Children's Book, the central character of which is a children's author with a philandering husband, recognisably inspired by Nesbit (or so I understand – I have abstained from reading it to date).

The possible novel I glimpsed in those few pages of Wells's autobiography was one in which his involvement with the Blands, and the conjunction of his writings and Edith Nesbit's, would provide a structure similar to the relationship between Henry James and George du Maurier in Author, Author. But I soon discovered from Julia Briggs's excellent 1987 biography of Nesbit, A Woman of Passion, that Edith's interesting story began long before she met Wells, while his own continued long after they became estranged, so their relationship could be only one episode in a novel about him

I already knew something of Wells's life from writing literary criticism about his work, but the more deeply I looked into it the more astonishingly rich in human and historical interest it appeared. Beginning inauspiciously (he was the son of unsuccessful shopkeepers and was apprenticed to the drapery trade at the age of 14), it stretched from 1866 to 1946, a period of global political turmoil, including two world wars, in which he played a public role. The bibliography of his published work contains some 3,000 items, including more than 100 books. He met and conversed with nearly every well-known statesman and writer of his time, and in his science fiction and speculative prose he foresaw the invention of, among other things, television, tanks, aerial warfare and the atom bomb. He made a strenuous effort to direct the Fabian Society towards his own idiosyncratic model of socialism (an updated version of Plato's Republic), nearly destroying it in the process, and worked selflessly if vainly all his life for the cause of world government. His Outline of History, published in 1920, was an ambitious attempt to "teach the peoples of the world . . . that they are all engaged in a common work, that they have sprung from common origins, and all are contributing some special service to the general end". It was a global bestseller.

"Thinking people who were born about the beginning of this century," George Orwell wrote in 1941, "are in some sense Wells's own creation." Between the wars, however, his influence gradually declined, along with the quality of his writing. The triumph of literary modernism in the 1920s made his work look old-fashioned, and the novels that have retained classic status, such as The Time Machine, The War of the Worlds, Tono-Bungay and The History of Mr Polly, all belong to the first 15 years of his long literary career. His mind remained fertile with new ideas – in the late 30s, for instance, he proposed something he called the "World Brain", an enormous bank of human knowledge stored on microfilm and transported free to users by aeroplane, which needed only the invention of the microchip to resemble the internet – but the world paid diminishing attention to them. There was pathos in his own sense of this neglect in his last years, and in his deepening pessimism about the fate of the human race, epitomised in the title of his last book, Mind at the End of Its Tether.

Wells was also a prophet of the sexual revolution of our own era. He believed in free love and practised it tirelessly. He was married twice to women he loved, but neither of whom satisfied him sexually, and had several long-term relationships, as well as innumerable briefer affairs, mostly condoned by his second wife, Jane. Of particular interest because of the scandal they aroused were his relationships with three young women half his age: Rosamund Bland, the secretly adopted daughter of Edith and Hubert Bland, who was actually fathered by Bland on Edith's companion and housekeeper, Alice Hoatson; Amber Reeves, a brilliant Cambridge undergraduate, also the daughter of prominent Fabians; and Rebecca West, whom he invited to his Essex country house in 1912 to discuss her witty demolition of his novel Marriage in the feminist journal The Freewoman, a meeting that led in due course to the birth of Anthony West on the first day of the first world war, and a stormy relationship that lasted for some 10 years. Reeves also became pregnant by Wells, by her own desire, with dramatic consequences. There were interesting liaisons with the novelists Dorothy Richardson (who portrayed Wells in her novel sequence Pilgrimage), Violet Hunt and Elizabeth von Arnim. Then there was Moura, Baroness Budberg, a Russian aristocrat who survived the Russian revolution as the secretary and probably mistress of Maxim Gorky and with whom Wells slept when staying in Gorky's flat in Petrograd in 1920. They met again after Jane's death in 1927. Moura was the great love of his later life and his acknowledged mistress, but refused to marry or cohabit with him. Wells has the reputation of being a predatory seducer, but in all the relationships I investigated, with the possible exception of the always inscrutable Moura, he was initially the pursued rather than the pursuer.

Sexuality, the most private and intimate aspect of a person's life, presents a challenge for the writer of a biographical novel. It wasn't a problem for me in the case of Author, Author, because I share the view of most of James's biographers that he was a celibate bachelor who repressed or sublimated his inherent homosexual tendencies. Wells was, in this respect, as in others, the antithesis of James. Fortunately for my purposes, he wrote a secret "Postscript" to his 1934 autobiography, about his sexual life, to be published after he and the women mentioned in it were dead. It eventually appeared, edited by his son Gip, in 1984 under the title Wells in Love. This gave me the essential facts about the major relationships in his life, and a large number of minor ones, as well as invaluable information about his sexual development in childhood and adolescence. It contains only hints of his proclivities as a lover, but these could be supplemented from other sources, especially his letters to West.

Like most students of Wells I came to the conclusion that he was riven with contradictions in principle and practice, but that he was also one of the most interesting and prodigiously talented figures in 20th-century cultural history. The main problem for me was to find in a mass of fascinating material a novel-shaped story, by which I mean a story that has more cohesion and patterning than the faithful chronicle of a life can usually provide. Gradually it emerged: a frame story of Wells's last years, 1944-46, set mainly in his Blitz-battered Regent's Park house, in which Rebecca West, Anthony West and Moura figure prominently; and between these two bookends, the story of the most interesting part of Wells's life, from childhood to the mid-1920s, recalled by him in a variety of discourses, first person and third person, following the sequence of the most important women in his life, showing how they affected, fed into, disrupted, and at times threatened to destroy his career as a writer and public man.

The biographical novel, a type of prose narrative which uses novelistic methods to tell a story about a real person's life, has become increasingly popular with writers and readers of literary fiction in recent decades, and lately has been the subject of controversy. In January William Skidelsky argued in the Observer that a concentration on historical reality stops writers using their imagination. A week later Antony Beevor attacked the trend from the opposite direction in a talk to the Royal Society of Literature on "The Perils of Faction". One of the many perils he identified was that "when a novelist uses a major historical character, the reader has no idea what he or she has taken from recorded fact and what has been invented in their re-creation of events."

I understand the concerns of Skidelsky and Beevor, but there are many different ways of combining fact and fiction, and each example must be judged on its own terms. Some bio-novels, for instance, put their historical characters into situations that they never actually experienced, or imagine encounters between historical characters who never met, sometimes in a comic, carnivalesque mode. I am drawn to the kind based on the known facts about the subject but which uses fictional methods to explore the gaps between them, including the subjective experience of the persons involved.

Through novelistic techniques that evolved in the 19th and 20th centuries – especially "free indirect style", which combines third-person narration with a character's inner voice, and the alternation of this with scenes of dramatic dialogue and interaction – a bio-novel can give a more vivid sense of a person's life as lived than the discourse of biography, in which the voice of the biographer predominates and the narrative content is determined by the available evidence. The novelistic method involves inventing – or, as I would prefer to say, imagining – innumerable small units in the continuum of represented experience. As Beevor observes, there is no way the reader can tell which they are. But as long as they are compatible with the factual record, and the book is presented and read as a novel, not as history, no harm is done, and something may be gained. Bio-fiction does not pretend to replace biography, but complements it, offering a different kind of interpretation of real lives. By putting himself imaginatively inside the consciousness of a historical individual the novelist can contribute to the understanding of biographical "facts".

The episode of Wells's life that required me to use most imaginative reconstruction was his affair with Rosamund Bland. Few hard facts are known about it. It began probably at or near Dymchurch, where the Blands had a holiday house, near the Wellses' home in Sandgate, in the summer of 1906, when Rosamund was a buxom, flirtatious young woman of 19, secretary of the newly formed group of young Fabians known as the Nursery, and very much under HG's spell. According to Wells's own brief, slightly ashamed account in the Postscript, he "never found any great charm in Rosamund", but "she talked of love and how her father's attentions to her were becoming unfatherly", so he decided to protect her from incest by possessing her himself. In this he was encouraged by her natural mother Alice, "who had a queer sort of liking for me". Hubert got wind of the affair and used it to blacken Wells's character among the senior Fabians later that year at a critical moment in his campaign to reform the society. Relations cooled between the two families but there was no permanent breach until, at some subsequent date, Wells and Rosamund were intercepted by Bland on Paddington station in the act of going off together – "for a dirty weekend in Paris", according to her sister-in-law's later testimony. By some accounts the enraged father, an amateur boxer who used to spar with Bernard Shaw, thumped Wells before dragging his errant daughter home.

Julia Briggs usefully pointed out that Wells may have planned to travel from Paddington to Plymouth to take one of the transatlantic liners across the Channel – less conspicuous than the usual routes. She also believed the incident must have happened some time shortly after 4 March 1908, because of a surviving letter from Rosamund to Jane Wells of that date, which begins:

Dear Mrs Wells,

Of course you have an invitation to the Nursery lectures. I wouldn't think of sending you a ticket. It never occurred to me to write and ask you because I thought you would understand that you were to come if you wanted to. I'm so sorry you aren't coming to our dance on the 20th. I thought I might have had an opportunity of talking to you a little bit.

Briggs asserted: "it is virtually impossible that Jane Wells would have been asked to a dance at Well Hall after the event [at Paddington]." With this I had to agree, but it created a serious problem for the cohesion of my novel. As Briggs was aware, Wells began his affair with Amber Reeves in the spring of 1908 – in fact during her Easter vacation, when she was preparing for the second part of her tripos examinations. It was the culmination of a mutual attraction, cloaked by a kind of tutorial relationship, which had developed at an accelerating pace that year – one of the great passions of Wells's life, and his most daring experiment in free love, which lasted for nearly two years until he very reluctantly agreed to end it. Why on earth would he go off for a dirty weekend with a girl he never really cared for, a few weeks before he and Amber became lovers? How could I make this psychologically plausible, and not utterly discreditable? The problem baffled me, and blocked the progress of my novel, until I suddenly saw the answer. Because the Blands used to hold dances at Well Hall, Briggs assumed that "our dance" in Rosamund's letter referred to such an occasion, but it was much more likely that it referred to a Fabian Nursery dance, to which Jane and HG had been invited as members of the executive. Because the archive of the Nursery held at the LSE doesn't begin until 1910, it is impossible to verify that they held a dance on 20 March 1908, but Patricia Pugh's history of the Fabian Society, Educate, Agitate, Organize, confirmed that the Nursery did indeed hold dances in the early years, which was good enough for me. I felt free to place the Paddington episode in the early summer of 1907, a much more plausible date for several other reasons. Rosamund's letter has exactly the wistful tone of someone who would like to heal a breach with a former friend.

Of course I could have ignored Briggs's dating of the Paddington incident when I first encountered it, and placed it at a different time – very few readers would have challenged me. But that would have been to break the rule I set myself: to respect the known facts. When the different documentary sources I consulted gave different versions of the same event, I favoured the one that seemed most plausible to me as a novelist. In the Postscript to his autobiography, Wells describes his third visit to Russia, undertaken primarily to interview Stalin, in 1934. He asked Moura, who had lived independently in Europe since she parted company with Gorky in 1928, to accompany him. She refused, saying she dared not return to Russia, and that she had to visit her children in Estonia, where they arranged to meet on his return journey. He took his son Gip as companion instead. Visiting Gorky in his dacha outside Moscow, Wells was stunned to discover that, unknown to him and contrary to her own accounts of her movements, Moura had stayed with Gorky three times in the past year, most recently only a week before his own visit. Wells felt betrayed and described vividly how he was plunged into paroxysms of jealous rage. He set off alone for Tallinn, determined to confront Moura with her deception.

In HG Wells: Aspects of a Life (1984) Anthony West asserts, naming Gip as his source, that Wells and his son deduced between them that Moura must be a spy working for Russian intelligence, that she had been planted on him at the very beginning of their relationship in 1920 and had been reporting on him ever since. When Wells accused Moura of this in Tallinn she admitted it. She told him that it was the only way she could have survived the revolution and that, "as a biologist, he had to know that survival was the first law of life." According to West, although Wells patched up their relationship, he never recovered from the disillusionment, and it was the underlying reason for the misanthropy of his last years.

West's account is repeated by John Gray in his new book, The Immortalization Commission, and Gray discussed Wells's and Moura's relationship in an essay for Review, published on 8 January. Without Gray's endnote reference, readers of that piece would perhaps assume that the story came from Wells's Postscript. It does not. Wells gives a very detailed account of his showdown with Moura in Tallinn – it is the one dialogue scene in my novel hardly a word of which I had to invent – and at no point in it, or anywhere else, does he accuse Moura of being a spy, only of being "a liar and cheat". West's book is a mine of information but he is not always reliable, and in this instance I have followed Wells's account. If West's version were true, why would Wells give a different one in a work to be published after he and Moura were dead? I find it hard to believe – and I would have found it hard to render in my novel – that he received Moura's frank admission in 1934 that she was a Russian spy who had all along exploited him out of self-interested motives, but that nevertheless he soon resumed a sexual relationship with her, begged her to marry him, and maintained that she was one of the few women he truly loved. Also her daughter Tania recalls in her memoir, A Little of All These, that Moura asked her in June 1936 to tell HG that she had been taken ill in Paris when in fact she had gone to Moscow to visit the terminally ill Gorky. Moura would surely not have bothered with this deception if two years earlier she had confessed to being a regular visitor to the Soviet Union in the pay of OGPU.

It would be surprising if Wells, knowing something of Moura's life in revolutionary Russia, never suspected that she had been compromised into acting as an agent for Russian intelligence, but I take the view that he suppressed or was in denial of this as a possible explanation of her attachment to him, and in my novel it surfaces only towards the very end of his life. Admittedly, in this position it helps to make my narrative novel-shaped.

Wells's golden age, by DJ Taylor

On Christmas Day 1903 HG Wells arrived in the French resort of St Jean Pied de Port, where his friend George Gissing lay dying. Welcomed into the sick man's bedroom, Wells immediately began to find fault with the nursing arrangements, insisted that his fellow-novelist – delirious and running a high temperature – was being "starved", force-fed him broth and, according to Gissing's widow, helped to kill him. To compound this exercise in insensitivity, Wells then put Gissing's last words into the mouth of Uncle Ponderevo in Tono-Bungay's celebrated death-bed scene.

It is difficult not to find something rather symbolic in this catalogue of impulsive gestures and rock-solid confidence in the matter of one's personal judgment: a neat little metaphor not only for the barnstorming way in which Wells conducted his life, but for the art which was that life's justification. If the majority of his novels, let alone the socio-political musings of his later period, were directed at the burning question of The Way We Live Now, then their guiding principle is the title of a second Trollope novel: He Knew He Was Right.

The first area in which Wells claimed specialist knowledge was the future. It is not an exaggeration to say that the half-dozen "scientific romances" published at the fag-end of the Victorian era shaped the thinking of an entire teenage generation. As Orwell once put it: "There you were, in a world of pedants, clergymen and golfers . . . and here was this wonderful man who could tell you about the inhabitants of planets and the bottom of the sea, and who knew that the future was not going to be what respectable people imagined."

The reference to "respectable people" is significant, for the novels of Wells's great period – essentially the first decade of the 20th century – are, at bottom, terrific assaults on the idea of "respectability" and its devitalising effect on the teeming lower-middle-class world from which he came. The heroes of The History of Mr Polly (1904) Kipps (1905) and Tono-Bungay (1909) are trying desperately to escape either their origins or the chains that hold them down. Kipps, a draper's apprentice, uses an unexpected legacy to try to turn himself into a "gentleman". George Ponderevo in Tono-Bungay hitches himself to the wagon of his uncle's patent-medicine scam. Mr Polly, trapped behind his Kentish shop-counter, fakes his own death. Each is hamstrung not by his mental horizons but by the stultifying hand of convention, class consciousness and the habit of judging every step you take by the yardstick of what other people might think of you.

In strict taxonomic terms, Wells is the link between the cramped lower-bourgeois interiors of Dickens and Thackeray's early work and Orwell's 1930s fiction. The scene in Coming Up for Air (1939), in which George Bowling revisits the Thames Valley town of his upbringing and finds his now-decayed first love working in a tobacconist's shop, is sharply reminiscent of a key passage in Mr Polly. But there's a difference between Kipps and Ponderevo and some of Thackeray's tuft-hunting arrivistes, and it lies in the fact that Wells actively sympathises with his heroes. He may mock Kipps's ambitions, he may insist that his aspirations are not worth the having, but in the end class solidarity always wins out, if only because Kipps, like Polly and Ponderevo, is a backward projection of Wells himself, minus the genius – a harmless, averagely accomplished lower-middle-class boy, condemned to the drudging, joyless life of the shopkeeper's assistant, until money sets him free.

Or not, as the case may be. For Wells's other great theme is that money is no help in negotiating the complex obstacle course of early 20th-century English social life. His moral is the moral of Great Expectations brought forward into the Edwardian age: don't throw over the class you were born into; don't imagine that the process of "bettering yourself" won't involve huge amounts of moral compromise and self-delusion. The great enemy in every book from Love and Mr Lewisham (1900) to Ann Veronica (1909) is what Wells calls "the ruling power of this land, Stupidity". Against this he envisions a world in which people will behave better to each other, a world in which honest aspiration and fellow-feeling won't automatically be snuffed out by snobbery and hidebound tradition, a world in which "equality" is not a dirty word. At the same time, it is self-evidently a world in which people like HG Wells are allowed to luxuriate and prosper. On one level, Kipps is an exposé of an outdated social system, of a life based fundamentally on the principle of fooling yourself, but is also, you imagine, an apologia pro vita sua.

What remains is a kind of sentimental realism, in which the happy endings, such as they are, altogether fail to disguise some of the genuine horrors that lurk behind the wainscoting of the average early 20th-century parlour: the out-of-work drapers' assistants quietly starving to death in library reading rooms, and tubercular journalists coughing their guts out in rented garrets. Wells was never a realist in the textbook sense, and the brute matter-of-factness of a contemporary American such as Theodore Dreiser was denied him both by the tradition in which he wrote and, even more important, by the view that he took of the world. In his essay "Wells, Hitler and the World State" (1941), Orwell proposed that Wells was simply "too sane" for the mid-century landscape of marching armies and lofted flags. By the time of his death in 1946 Wells openly despaired of the state of the world in which he found himself; but in the golden years before the first world war, he was merely exasperated, confident that a little more collective action, a little more personal resonance, could change things for the better.

"I don't suppose there ever was a chap quite like me before," Kipps tells his wife. In fact, this is a red herring, for the English novel of the previous century is full of bewildered socio-economic migrants of the Kipps sort, ripe to be exploited by the unscrupulous predators of the gentlemanly drawing room. On the other hand, no previous novelist had Wells's ability to decode the assumptions on which a certain kind of middle-class life was based. These early forays into social realism are, consequently a number of things all at once – disguised autobiographies, economic clarion calls, successful attempts to extend the English novel's social range. But they are, above all, horribly funny books, written by a man who still believed that the most effective way of attacking something was to laugh at it.