I have been stumping around the country talking about Dickens in this bicentenary year; afterwards, I invite questions from the audience, and at some point, someone – usually a woman – will ask me how I can possibly admire a man who treated his wife so atrociously.

Eating humble pie on behalf of my hero, I admit that, yes, Dickens fell out of love with his wife, Catherine; he became surly and critical towards her, then, after 20 years of marriage and 10 children, and without informing her in advance, had the marital bedroom divided in two, soon after demanding, on no particular grounds, a legal separation from her.

She left the house, and he never saw her again as long as he lived. He wrote a baffling statement about the situation, which was printed in the Times and elsewhere, and another letter for private circulation in which he described her as an incompetent mother and possibly afflicted with some mental disorder. I confess, again on behalf of Dickens, that this letter is a disgraceful document.

I then note that nonetheless he gave her a house, a carriage and a substantial annual income of £600 to be continued as long as she lived, and that he encouraged their children to see her whenever they wanted to. I add that, despite his erratic and unkind behaviour, he didn't beat her or starve her; neither, as far as I or anyone else is aware, did he have any extramarital affairs before their separation. In other words, he behaved like many men who have fallen out of love with their wives, except that he was Charles Dickens and everything in his behaviour was proportionately magnified.

I rarely have the feeling that my questioner is satisfied by my reply; Dickens the domestic monster has become part of the intellectual landscape, along with some increasingly lurid speculations about his sex life. A hair-raising version of these is quoted by Michael Slater, at the beginning of his deliciously dry and compulsively readable new book: "Charles Dickens," wrote Victoria Coren in the Observer in 2005, "the man who committed what was in his lifetime considered incest with his wife's young sister. Poor Mrs Dickens was banished to a separate bedroom while her husband conducted many affairs, until finally he abandoned her and took their children with him. Nice guy." Apart from the banishment to a single bedroom, there is not a word of truth in any of this.

What happened, or as close to it as we will ever get, is laid out with characteristic meticulousness by Slater, the Dickens polymath whose Dickens and Women remains one of the most illuminating books on the infinitely complex personality of our greatest novelist. The Great Charles Dickens Scandal has the form of a detective story, carefully and with amused scepticism reprising all the evidence concerning the woman – not his sister-in-law – with whom he was involved until his death in 1870, Ellen Ternan. Dickens met the 17-year-old in January 1858; she was to take over a small part in his sensational production of The Frozen Deep, the Arctic melodrama Wilkie Collins had written for him. Dickens, it seems, instantly fell in love with Ternan. Meeting her sounded the death knell for his marriage; this is when he divided the marital bedroom in two.

After his separation from Catherine, and not, as far as we know, before, he pursued his relationship with Ternan, which (though again, there is no hard evidence) it seems safe to assume developed into a full affair. It was of the utmost importance to Dickens that no one should know of this relationship. He had a morbid dread of loosening the bond between himself and his readers, which he rightly saw as being "personally affectionate and like no other man's".

But it was more than that; it was the mainspring of his life, and he allowed nothing to imperil the relationship. He therefore set about, with all the vigour and ingenuity that was his to command, concealing the very existence of Ellen Ternan. Among his many precautionary measures, which included adopting aliases and inventing codes, he burnt all his papers, all his documents and every letter he had ever received. He knew that he was a symbolic figure for millions of people, and he was determined to protect them and himself from any uncomfortable facts.

The world's attempts (so far only partially successful) to break through Dickens's firewall is the subject of Slater's book, the saga of how, in close cahoots with members of the Dickens family, an increasingly embattled hello-clouds-hello-sky Dickens Fellowship – intermingling, as Slater says, "meat-teas for poor children and sewing-bees to make clothing for the poor with topographical investigations, speculations about the intended ending of Edwin Drood, and lectures on Dickens's life and works" – attempted to maintain an impossibly spotless image of their hero.

The publication in 1928 of a novel, This Side Idolatry, by Carl Bechhofer Roberts, which included a number of sensational elements of the Dickens story, along with some wild speculations, unleashed further revelations about Dickens's intimate life. This was shocking, even in the 20s, to a public to whom he was still something of a god. His reputation as a paragon of virtue became a matter of national significance, culminating in a circulation war between the Express, which published the enthrallingly explicit but still contested confidences of Ternan to her parish priest, at the same time as the Mail was serialising the touching Life of Our Lord that Dickens had written for his children.

Accusations and counter-accusations polarised views of Dickens: was he, the headlines demanded, a monster of depravity or a secular saint? A more forensic approach led, a decade later, to the discovery of the first real hard evidence: in 1943, John D Gordan used infrared to discover concealed references to Nelly in Dickens's letters. Slater's chapter "Enter the Scholars" is thrilling; the quest for Dickens's secret life becomes the sort of manhunt Dickens himself had so often written – the criminal pursued by implacable detectives.

All this seems impossibly remote to us. In the 1980s, Kathryn Hughes remarked that if it could be proved beyond reasonable doubt that Dickens had a sexual liaison with Ternan, the effect upon the public mind would be "like finding that Father Christmas had been to a brothel". Nowadays, we are more inclined to admire him for having had a sex life than to condemn him for it; at the very least we feel that it makes him more human. But the truth of the matter is that we still know very little about his erotic life. Did he have sex when he accompanied the erotically liberated Wilkie Collins to Paris? What exactly did he get up to when he roamed the London criminal underground by night? What precisely was the nature of his relationship with Ternan? Despite the highly plausible speculations of, among others, Claire Tomalin, we still do not know. As Michael Slater, towards the end of this wise, witty and highly entertaining volume, observes, echoing Edmund Wilson: "There is … much greater recognition now … of just how odd a case he actually was and nowhere more so than in his relations with women." There are no simple answers to the mystery of Charles Dickens; he remains richly and eternally unfathomable.

• Charles Dickens by Simon Callow is published by Harper Press.