Henry James was the originator in ­English of novel-chauvinism, the idea that the extended prose fiction is, as he put it, “the book par excellence”. Between 1871 and 1904, during which time he published 19 novels, including The Europeans, Washington Square, The Portrait of a Lady and What Maisie Knew, plus novellas such as Daisy Miller, The Aspern Papers and The Turn of the Screw, James also wrote dozens of reviews, essays on “The Art of Fiction” and “The Future of the Novel”, and half a dozen stories about the novelist’s earthly tribulations and posthumous mistreatment. In the words of a later novel-chauvinist, F R Leavis, James set out to accomplish “a general full recognition among the educated that creative talent – creative genius – was at least as likely to go into the novel as into any mode of art”. In this effort, James employed all the big rhetorical guns, not just French tags (“par excellence”), but capitals and superlatives. In the closing words of his preface to The Ambassadors, which he considered “the best, ‘all round’, of my productions”, he stated that “the Novel remains still, under the right persuasion, the most independent, most elastic, most prodigious of literary forms”.

The chauvinist position had a limiting effect on his relationship with non-novelists. T S Eliot, a fellow uprooted American and like-minded elegist for the unlived life, looking back on the arid years of his apprenticeship, said that he “learned something, no doubt, from Henry James”. That “something” went a long way, in terms of borrowed images and allusive phrasing. But Eliot felt he would have learned more if not for James’s “exclusive concentration on his own kind of work”. James moved through the world in a pair of novel-blinkers. Confronted with the volumes of Browning’s The Ring and the Book, he claimed, in a 1912 lecture, to experience “the sense, almost the pang, of the novel they might have constituted”. He called this phantom novel “a work of art . . . smothered in the producing” – hardly an orthodox way to mark the hundredth birthday of a poet.

But, for all this, the most prominent new publications during the centenary of James’s death, on 28 February 1916, do not relate to his fiction: a Library of America edition of his autobiographical writing, edited by Philip Horne, and Oliver Herford’s Henry James’s Style of Retrospect, a diligent and minutely argued study of the “late personal writings”. It is true that the Library of America has already produced 11 volumes of James’s fiction; that Cambridge University Press is in the process of doing a scholarly version of the same thing in 34; and that the James industry has already yielded writing on every inch of that work. Still, for almost three decades there has been a growing critical and editorial engagement with his non-fiction writing from the decade after his “major phase” novels – The Wings of the Dove, The Ambassadors and The Golden Bowl – during which he published virtually no fiction. This work, amounting to a “second major phase”, includes the contentious travelogue The American Scene, an account of his 1904-05 tour of the much-changed United States he had left for Europe 30 years earlier; the long explanatory prefaces to the 24 volumes of the New York Edition of his fictions; the essays collected in Notes on Novelists (1914); and the autobiographies A Small Boy and Others (1913) and Notes of a Son and Brother (1914), which are printed alongside essays, notebook entries and an uncompleted third instalment in the Lib­rary of America volume.

If the late novels, with their snaking, ­nuance-laden sentences and relentless tracking of the characters’ inner lives, show that James was in at the beginning of literary modernism, the later non-fiction shows why he might be considered modern and even modish. The recent writing on him as a novelist talks of impressionism, vagueness, religious experience, melancholia. In the non-fiction, we meet James the Freudian, the explorer of family dramas and selfhood, the proto-psychogeographer given to flâneries, a figure whose descendants are neither self-conscious Jamesians such as Alan Hollinghurst and Cynthia Ozick nor self-declared modernists such as Eimear McBride and Tom McCarthy, but rather the likes of Karl Ove Knausgaard and David Shields, who embrace the literary potential of the memoir and the essay. In a letter written in 1908, when the prefaces had followed The American Scene in monopolising his attention, he spoke about being taken away from “‘creative’” work, throwing the

primacy of the novel into doubt many decades before somebody offered the first graduate seminar in “literary non-fiction”.

In his book Reality Hunger (2010), a plea for realistic art which proceeds from the conviction that the novel, far from being the only game in town, has long since had its day, Shields fingers James as an enemy – perhaps the enemy. It was James, he alleges, who took the novel away from its exciting “mongrel” origins, in documentary, topographical writing and autobiography, and asserted that it “must be the work of the imagination alone”. It isn’t clear where Shield got this idea, which is equally wrong about what James rejected and what he embraced.

Tony Tanner, in his book Henry James and the Art of Non-fiction (1995), pointed out that James insisted from the start “on the truth and reality of the novel; almost . . . on the non-fictionality of fiction”. Any reader of James’s prefaces or his working notebooks will know him as a serial plagiarist of dinner-party anecdotes. As if to win Shields’s approval, James said The Princess Casamassima, his 1886 novel about Victorian anarchism, emerged “from the habit and the interest of walking the streets”. In “The Art of Fiction” (1884), his own Reality Hunger, James urged novices: “Write from experience, and experience only.”

James gave more thought than David Shields could ever dream of doing to the problem of accommodating reality in writing. He believed that the “most fundamental and general sign of the novel, from one desperate experiment to another, is its being ­everywhere an effort at representation” – and representation for James is founded on a troubling conflict of priorities which he wanted to resolve.

***

In their feisty exchange of letters in 1915, H G Wells advised James that life “must not be whittled or distended for pattern’s sake”. But James did not need telling. It had been his life’s ambition to give a sense of reality through what he called form, which in his work usually meant that characters occupied a clear role in the book’s moral or conceptual structure, and that the plot would unfold in dramatic scenes, not through exposition or overt meditation.

It is understandable that Wells and others, such as E M Forster in Aspects of the Novel, might have thought that James favoured “pattern” over “life”. In his preface to The Tragic Muse, James had written that a picture without composition “slights its most precious chance for beauty”, and invoked by way of negative example “such large loose baggy monsters” as War and Peace, Thackeray’s The Newcomes and The Three Musketeers. Pre-empting the reply that these books exhibited “life”, he said it was the wrong kind – wasteful, otiose. What, he wondered, do these books “artistically mean”? For James, the Victorian novel was guilty on this count, not just Thackeray, but also Dickens, Trollope, Hardy and even George Eliot, who in Middlemarch provided a “treasure-house of details, but an indifferent whole”. The English novelist did not excel in what James called the “literary part of the business”. For examples of “what composition, distribution, arrangement can do”, he said, you have to “go elsewhere”: namely to the French and their superior display of audacity, neatness, acuteness, intellectual vivacity, fine arrangement of material, and so on.

But the French approach carried its own vices. It might not produce books that were large, loose and baggy, but it did produce books that, as James put it in an 1893 essay on Flaubert, were unconsoled, unhumorous and unsociable. In Zola’s Rougon-Macquart sequence, which considered a French family through heredity and environment, method became “almost the only thing we feel”. Here James praised the English for being “strong, genial, and abundant” – for being at home in the “moral world” and holding “their noses close . . . to the texture of life”. The novel’s tendency to “appear more true to its character” when it bursts its mould was, he wrote, its “high price . . . as a literary form”. Pattern without life was no better than life without pattern.

The reconciling of these opposites, achieving “a selection whose main care . . . is to be inclusive”, produced in him a sense of novel-anxiety, even defeatism. He lamented that there is “no art at all . . . that is not on too many sides an abject compromise”.

David Shields recognised James as a predecessor for his committed thinking on this problem, but not as a novelist practising an unmixed sort of non-fiction. In Reality Hunger, he identifies John Cheever’s journals, F Scott Fitzgerald’s misery memoirs and E M Forster’s Commonplace Book at the expense of their well-kempt novels as the work that gets closest to their experience of reality. But James should be Exhibit A in Shields’s case. He wrote in all of Shields’s beloved observational and introspective modes and did so when handling subject matter that Shields, too, has treated in non-fiction: American capitalism, the importance of reality in literature, his own life. James was the first novelist to publish, in A Small Boy and Others, a work of personal history that didn’t resemble the “Life” of a statesman (Trollope) or a saint (Tolstoy), pipping Proust by seven months – and besting Proust, by Shields’s criteria, because he did not feel the need to invent. Reminiscences of schooldays, visits to the theatre and museums, the food in the family larder, the “sweet taste of Albany”, came, James said, “from everywhere at once”. His amanuensis Theodora Bosanquet, in a vivid memoir printed as the appendix to Horne’s volume, recalled that “no preliminary work was needed”: “A straight dive into the past brought to the surface treasure after treasure, a wealth of material which became embarrassing.”

Embarrassing? Perhaps; but justified in the text itself, which, like Knausgaard’s My Struggle, is as preoccupied with remembrance as with memories. Early in James’s memoir, realising that he can only recall how he behaved during one long boyhood walk downtown with reference to another, and finding himself divided between the “still present freshness” of some memories and “my sense of perhaps making too much of these tiny particles of history”, he decides he will be guided by a rule:

. . . from the moment it is a question of projecting a picture, no particle that counts for memory or is appreciable to the spirit can be too tiny, and that experience, in the name of which one speaks, is all compact of them and shining with them.

Soon enough, as in Knausgaard, we return to voluminously subclaused sentences rich with evocation of a younger brother’s youthful callowness, escape through reading and dreams of future glory as a novelist.

James was glad to have chosen a “vast miscellany” that accommodated “everything”. But he still hoped the book would achieve “a grace of its own”, and a line from the opening paragraph of A Small Boy and Others implied a belief that, by some kind of internal logic or magic, it could achieve the synthesis similar to the one he sought in his fiction. To “knock at the door of the past”, he discovered, was “to see the world within begin to ‘compose’ with a grace of its own round the primary figure”.

In a notebook entry from 1905 printed in the new Library of America volume (the closest he came to a non-fiction credo) James dismisses the intense memories that were visiting him while writing The American Scene as “irrelevant strayings of the pen, in defiance of every economy”, before deciding that “to present these accidents” – to catch in full the unstaunched flow of recovered experience – “is what it is to be a master”. Oliver Herford explains that the critic Richard Poirier, reviewing an edition of James’s notebooks, insisted that James must have written “prevent”. Herford has seen the original. The S may be “cramped”, but an S it is. In a formulation that David Shields would feel obliged to cheer, Herford writes that the “stylistic accomplishment of the late personal writings may indeed be called a mastery of accidents” – that is, a masterly presentation and not a masterful prevention of them. But, like Wells and Forster, Poirier also seems to miss James’s perennial ambivalence towards pattern, and how he had always been concerned with presenting “accident”, but in such a way that – unlike in the baggy monsters, “with their queer elements of the accidental and the arbitrary” – it would yield meaning.

Why, you might wonder, did James not just stick to the independent, elastic, prodigious novel? That was the chink in his chauvinism: not an insistence that there were places the novel couldn’t go, but an openness to the virtue of non-fiction. Unlike Shields and Knausgaard – “just the thought of a fabricated character in a fabricated plot made me feel nauseous”, he writes in book two of My Struggle – James saw possibilities and not an ultimatum. He turned to non-fiction despite his belief in the supremacy of the novel, and he returned to the novel for the same reason. In 1915, in a letter to Wells, he wrote: “It is art that makes life, makes interest, makes importance . . . and I know of no substitute whatever for the force and beauty of its process.” And the 1908 letter where he placed “creative” between inverted commas also expressed the feeling that he “could really shed salt tears of impatience and yearning” to get back to novel writing.

For all the difficulties that the novel posed, there was one practitioner who offered James hope of a resolution: Balzac, “the father of us all”. After despatching Tolstoy and Thackeray in his preface to The Tragic Muse, James talked of his delight in “a deep-breathing economy and an organic form” – terms that encode his belief in a fusion of the novel’s warring priorities. This is what Balzac had achieved: a “solidly systematic” literary composition, combined with “free observation” and “personal experience”. And in The Portrait of a Lady and The Ambassadors, James felt that he had managed the same. Others agreed. In his essay “Henry James: an Appreciation”, published in 1905, Joseph Conrad, who along with Ford Madox Ford sought to emulate Jamesian form, noted that the reader is “never set at rest” by his novels. “His books end as an episode in life ends. You remain with the sense of the life still going on . . .”

A century later, James offers an escape route from our present bind. His personal writing supports the claims for non-fiction, while the obstinacy of his novel-chauvinism

serves as a rebuke to anti-novel sloganeering. Working on his final novel, The Ivory Tower, and returning to questions of “matter” and “manner”, form and life, he wrote:

I come back, I come back, as I say, all throbbingly and yearningly and passionately, oh, mon bon, come back to this way that is clearly the only one in which I can do anything now . . .

That The Ivory Tower avoided treating the Great War does not mean that James considered the novel a form unequal to this task. That he died before completing it was not a Freudian slip.

Henry James: Autobiographies , edited by Philip Horne, is newly published by the Library of America