I was once told by a publisher that a novel I'd submitted "lacked redemption". I could not contain my excitement. At last, I said to my agent, I'd written, and been recognised for writing, a bad boys' book. She looked at the carpet for what seemed like hours. "I think what they're trying to say," she replied, when the silence could go on no longer, "is that they don't like it."

You take the kicks and the halfpence, to borrow a phrase from Conrad. But I wondered what the publishers in question would have said had the manuscript of Kafka's The Trial landed on their desk. Show me the redemption in K's dying for no reason with a knife twisted twice in his heart. "Like a dog!" K says as the novel ends, "as if he meant the shame to outlive him." And what about such splenetic, unconsoling and wilfully damnable novels as Rabelais's Gargantua and Pantagruel or De Sade's The Hundred and Twenty Days of Sodom or Henry Miller's Tropic of Cancer or Céline's Journey to the End of the Night, or Philip Roth's Sabbath's Theater, or even Conrad's The Secret Agent – a work the author himself found it necessary to rescue from the accusation of "moral squalor". Whence redemption as a measure of literature's worth, and how to justify it given how little in the way of atonement on the Christian model, ie deliverance from sin; and how little in the way of intelligibility on the rationalist model, ie deliverance from fragmentation, so many of the world's great novels countenance?

I shouldn't pretend not to understand what in fact I understand only too well. I was a "reverence for life" man – "see life steadily and see it whole" – in my days as a lecturer in English lit. We are, I argued, if not exactly "saved" by reading, at least partially "repaired" by it: made the better morally and existentially. To those who found that idea fanciful I would put the question: when were you last mugged on the Underground by someone carrying Middlemarch in his pocket? We read to extend our sympathies, to see ourselves in others and others in ourselves, to educate our imaginations, to find liberation from the prison of the self, to be made whole where we are broken, to be reconciled to the absurdity of existence, in short to be redeemed from flesh, the ego and despair. And even as I rehearse these platitudes I have difficulty letting them go. They are at the heart of the idea of a liberal education, and now is not the time, is it, to be reneging on that?

Let me be clear: substitute Tropic of Cancer for Middlemarch and I'd still argue against the likelihood of the bearer mugging you on the Underground. Reading literature remains a civilising activity, no matter that it's literature in which people do and say abominable things and the author curses like the very devil. What's at issue is how we describe the way the civilising works.

If we declare ourselves, as readers, to be on the side of life, the question has to be asked what sort of life we are on the side of. Life governed by the rules of respectability and fear? Life rounded at the edges with all the horror turned away from? Life seen whole and steadily with all the breakages and shaking taken out? I don't mean to set up false dichotomies. I would never say of those great writers whose work clearly falls outside the category of non-redemptive, even anathematising black-heartedness I am championing that they make us "feel good". Jane Austen's vision is a fraction from being a despairing one, her final chapters are dispensations of kindness, like the fifth acts of Shakespeare's comedies, in which we are spared bleakness by a hair's breadth, though we feel its presence all around. George Eliot's best novels simmer with a sometimes murderous frustration, no matter that her thwarted heroines can finally be said to make things a little less "ill for you and me than they might have been". If such dying falls allow us to sink for a while into a contemplative wistfulness, our nerves go on feeling frayed long after. No good writer ever merely cheered us up. But there's an unblinking stare into the darkness of things we have to go elsewhere to find. Jane Austen was made of strong stuff. She was too satiric for DH Lawrence's taste and too unforgiving for Kingsley Amis's, but you would still not call her hellish.

Céline: 'fascist and antisemite in his spare time'. Photograph: Lipnitzki/Roger Viollet/Getty Images

Other assumptions, besides a longing for coherence and reconciliation, lie behind the expectation that novelists must give their readers a bone of redemption to chew on. Chief among these being that a novel should be fair and well-balanced, a mirror of the likeable and acceptable, committing none of the attitudinal sins we refuse to tolerate on campus or in the progressive workplace, or in the marriage bed. Thus Lawrence was all but wiped from the consciousness of students of English literature in the 1970s as a consequence of Kate Millett and others loading him with the twin accusations of patriarchy and misogyny. Apologists for Lawrence have been so busy ever since proving those accusations unfounded that there's been little time to argue against their relevance in the first place. Patriarchy and misogyny are no more terms that belong to criticism of the novel than are, say, baldness or shortness of stature. A novelist might be a misogynist if he chooses. Ditto a misanthropist. Ditto a hater of straights or gays. Ditto an antisemite. It hardly needs arguing, you would think, that the more cautious we are grown in our transactional politics, the more grateful we should be when our novelists enable us, imaginatively – by way, if you like, of experimental play, as a sort of moral dry-run – to show contempt for them.

But it isn't only because it's a corrective to whatever version of decorum happens to be current that the novel of disrespect and malediction – there's no agreed term for the genre – matters. A novel by De Sade or Céline or Henry Miller has its own dynamic. Its ruffianism has no social function; it satisfies itself. And if it satisfies readers wearied with having to mind their language and their manners, that relief is only a small part of the pleasure. A small part of the exhilaration, I should say, because that in the end is what needs to be explained – how it is that the darker the view of life the more energised we are.

Nietzsche is the usual starting place for discussions of this sort, but I think Roth lays it on the line provocatively enough in Sabbath's Theater when he has his effrontuous, indecorous, needling hero Mickey Sabbath express an "uncontrollable tenderness for his own shit-filled life". "More defeat. More disappointment! More deceit! More loneliness! More athritis ... God willing, more cunt!" A list, if it is not to go on into indefinite provocation, as shit-filled lives do, that concludes with a self-congratulatory reflection. "For a pure sense of being tumultuously alive, you can't beat the nasty side of existence."

Roth has the best vocabulary of emotional tumult – I would go so far as to say he has claimed the word "tumultuous" for his own – of any writer living. Whether it's Oliver Twist wanting more gruel or Isabel Archer wanting more freedom, all heroes and heroines of literature are in want of something, but few crave degradation with more garrulous gusto, and more indifference to what we think of them, than Roth's – Mickey Sabbath being the greediest and most fuck-you of the lot. It's in the nature of reading a great writer, provided we don't erect artificial barriers of disapproval, that we should find such gusto infectious. The words lacerate and indulge, the drum beat of defiant self-excoriation grows louder and more insistent, Sabbath hurtles from low place to still lower like a Satan who has no power or influence to lose, and we are lying to ourselves, or we are over-guarded against unholy pleasure, if we don't admit we are made giddy by the descent. There is no necessity in literature to choose. We can ascend and descend. We can feel with Dorothea who would be a better person and we can feel with Sabbath who would be a worse. But for that pure sense of being riotously alive – the invocation of "purity" in such a context is of course a spiky joke at the expense of purists of every sort – it's Sabbath we turn to.

One can hear the French novelist Céline – fascist and antisemite in his spare time – in that hymn to life's nastiness. Roth is a long-time admirer. Not of the antisemitism – "I have to suspend my Jewish conscience," Roth wrote – but of the liberation he found in Céline"s novels. "I feel called by his voice," he went on. It's a fair guess that he was more than usually "called" by Céline at the time he was writing Sabbath's Theater. "In New York," muses Ferdinand Bardamu, Céline's insolent, shit-filled hero, lamenting the absence of concierges with their sordid gossip and all-purpose hatred, "they're cruelly lacking in this vital spice, so sordid and irrefutably alive …" A dozen pages later he is on the prowl for the horrors he can't live without. "To survive, I needed lecherous tonics, drastic elixirs." Sabbath would have nodded an acid assent to that.

The "vital spice" doesn't have to be sexual but it usually is. Nor does the sex have to be false or wretched or embittered, but if it's an energised refusal of redemption you're after, then sex gone wrong takes some beating. "When a woman lies to me," Karoo, the hero of Steve Tesich's great, neglected black comedy of that name, confides, "it's as close as I get to feeling loved. Whenever one of the women in one of my many short-lived love affairs faked an orgasm, I was always deeply moved … to think that she actually cared enough about my feelings to go to the trouble of faking."

Roth: 'the best vocabulary of emotional tumult'. Photograph: Bob Peterson/Time & Life Pictures/Getty Images

In their own individual, rancidly sardonic way these novels of which I speak are always funny. It infuriated Kafka when his friends didn't laugh at the stories he read aloud to them. But the blackest comedies can baffle readers not trained, or just unwilling, to recognise the comic in human extremis. It's obscene, rock-bottom laughter, disabused of all idealism, that provides the tonic Céline speaks of and the tumult Roth revels in creating. "Irrefutably alive" because wedded to life itself in all its messiness, shorn of consolation or illusion. At the last, as though refusing to give decency the satisfaction, Sabbath turns down the option of suicide. "He couldn't do it. He could not fucking die. How could he leave? How could he go? Everything he hated was here."

"To tell you the truth," Roth said in an interview published by La Quinzaine Littéraire in 1984, "in France, my Proust is Céline!" That's not difficult to understand. And it isn't just the swearing or the bleakness. The greater life he found in Céline is to be ascribed, I think, to the fractured, forever changing, never to be fixed and never to be true, picture of experience to which that writer, in every meandering sentence and impatient tussle with ideas, is committed. Proust's conscientious pursuit of sensation in all its subtle detail, as though memory is as much a moral as a sensory obligation, is not Céline's way. Truth for him is not to be found at the end of interminable, bed-ridden excogitation. It is caught on the wing, on a whim, at night, and as like as not in the street, where memory can be prompted by, but just as soon confused with, the soiled paper blowing about your feet.

To say which is to admit the sentimentality to which literature of this sort is inevitably prone. I mean the sentimentality of the poète maudit, the solipsistic self-love of the blasphemer. Anyone can do the nastiness of life; it's the rendering of the look and feel of the tumultuous that takes genius. "Men, days, things," writes Céline of a tropical hell in which Bardamu finds himself, "they passed before you knew it in this hotbed of vegetation, heat, humidity and mosquitoes. Everything passed, digustingly, in little pieces, in phrases, particles of flesh and bone, in regrets and corpuscles; demolished by the sun, they melted away in a torrent of light and colours, and taste and time went with them, everything went. Nothing remained but shimmering dread." You know a writer's good when you hear the vitality of his language and rhythms even in translation. Give up on our shit-filled lives or stay with them to the end, in the final resort what counts is evoking the brilliant, irrefutable pungency of their stench. And it's at this that Miller, too, excels. Remembering only the indecorous sex, we forget what a marvellously funny and vivid writer he was. Moldorf's "face is close to the carpet; the wattles are joggling in the nap of the rug … He falls on her lap and lies there quivering like a toothache. He is all warm now, and helpless. His belly glistens like a patent leather shoe. In the sockets of his eyes a pair of fancy vest buttons."

But yes, it's true that the temptation to wallow in misanthropy lies in wait for even the best. Much as I relish those last lines of Sabbath's Theater I have a feeling that perhaps I shouldn't. A bit too obviously naughty, are they, like Sabbath's just a little too preposterous occupation as obscene finger puppeteer? Miller's devil-may-care down-and-outness the same. And Céline, when the wind is not blowing hard enough around his feet, drifts a little too perfunctorily, "deeper and deeper into the night". It's easier to curse, when you're on a roll, than to bless.

Every form suffers its own inevitable surfeit. If George Eliot can be prolix in her moralising some times, and Lawrence over-insistent in his erotic sermonising, why can't these novels of drastic elixirs and lecherous tonics occasionally over-prescribe their poisons?

In Guy Ableman, the would-be scurrilous hero of my latest novel Zoo Time, I give mocking voice to such equivocations. A writer himself, once passingly successful, now not, he nurses an ambition to write a blackguardly novel on the Céline or Miller model if he can find a way to do it. He has already been likened to Rabelais and Apuleius, but he has been likened to Mrs Gaskell too. At school he studied the polite classics until a new teacher turned up and told him he'd been reading "novels for girls with migraines". You must read viscerally, boys, he counselled the class, "with your bowels". You don't forget advice like that.

So Guy is divided in his soul, a nice boy longing to lead a disorderly life, a tame novelist longing to be a feral one, a man who lives in his head longing to write with his bowels. He steals one of his own second-hand novels from an Oxfam bookshop. Doesn't that make him an outlaw? He plans an affair with his wife's mother. Doesn't that make him a sinner? Yes, he desires her, but the thought of a novel about a man who has an affair with his wife's mother excites him still more.

Does he have the stomach for it? And more to the point, do any of his readers, supposing there are any readers left? I wouldn't say that Zoo Time is a lament for the novel of gratuitous offensiveness. I wouldn't say it's a lament for anything. But the possibility is floated that it's over for serious novels of any sort other than those that salve our faint-heartedness.

People would rather have their porn soft – synthetic fur handcuffs, jiggle balls and smiles all round after – than face that tumble into exquisite death that writers such as De Sade and Bataille know to be eroticism's true consummation. In the same way they would rather hear Anne Frank aver that "In spite of everything I still believe that people are really good at heart" than read Primo Levi's gathering despair or the survivor Jean Améry refusing forgiveness and redemption. "Nothing has healed," he wrote towards the end of his life. And elsewhere: "Home is the land of one's childhood and youth. Whoever has lost it, remains lost himself."

There is no being reconciled to loss. What's gone is gone. What's suffered is suffered. But some novelists make it possible for us to stare at pain with bitter and derisive comedy, and because there is a part of us that values truth above illusion, we grab at that bitter comedy for dear life.