I can remember quite clearly how I first encountered DH Lawrence as a writer of something other than fiction. We were studying Hamlet at school, reading the expected lit-crit by AC Bradley, G Wilson Knight and, more fashionably, Jan Kott (Shakespeare Our Contemporary). But my teacher also nudged me towards a strange piece of writing by Lawrence called “The Theatre”, about going to see a production of Hamlet. Wanting to reduce the piece to exam-directed utility I didn’t know what it was or how it was meant to be read. Obviously it was about Hamlet (a “statement of the most significant philosophic position of the Renaissance”) but it was also a kind of story, a recreation of an actual experience and place.

The critical essays I had read up to that point all seemed like more diligent and accomplished examples of what I was reading them for, ie homework. There was no suggestion of homework about “The Theatre” and while this had an obvious appeal it also raised doubts as to the legitimacy and value of the piece. On reflection what I missed, I think, was the valorising dullness that pervaded so much of the criticism that came to define the study of English at university. The gap in enjoyment between the novels or poems and the stuff we were expected to read about them was so huge.

Until, that is, during the week devoted to Thomas Hardy when Lawrence came crashing in again and suddenly there was no gap. One moment he was pointing out, in his rather homely way, that Hardy’s characters “are always going off unexpectedly and doing something that nobody would do” and the next making metaphysical pronouncements about “the great tragic power” of Egdon Heath. I read Study of Thomas Hardy (1914) for the light it shed on Hardy, but it was also a revealing expression of who it was by: as much mirror as window.

Up until then non-fiction existed either as a wholly distinct discipline (history, say) or as a kind of stepladder to help one better get to grips with poetry and novels. These pieces by Lawrence represented the first glimpse of a more labile relationship between criticism and fiction, between the necessary restraints of academic discipline and the vagrant life of the mind. (Lawrence went further, rejecting the limiting life of the mind in favour of “a belief in the blood, the flesh, as being wiser than the intellect”.)

The combination of commentary and imaginative writing achieved full expression – or, in Lawrence-ese, was consummated – in Studies in Classic American Literature (1923) which remains one of the wildest feats of critical mapping ever attempted, not just of the main components of a national canon but the soul of an entire country. Those early pieces on Hamlet and Hardy were transformative for me. Forty years later the essays of Studies in Classic American Literature remain in constantly surprising proximity to my adult experience. At 25 Philip Larkin was too young to know whether his precocious claim that “no one who has really thrilled to Lawrence can ever give him up” had any validity, but it has proved accurate in my case.

Shortly before writing this essay I was staying at a friend’s place in Joshua Tree in California. Late one afternoon we hiked up a hill in the blazing sun to a trailer that had been abandoned by the owner. Inside there were mouse and rat droppings on the floor, the bed, all over the kitchen. The scattered remains of what had once been a domesticated life intermingled with the detritus of subsequent drug use to change the various rooms from a home into “horrible underground passages of the human soul”. Unchanged in millions of years, the surrounding golden landscape and deep blue sky were tremendous. On a picnic table outside the trailer were the brittle pages of a cover-less edition of the writings of Edgar Allan Poe. It was impossible to know what “ghastly story of the human soul in its disruptive throes” had unfolded here but, for me, the mystery itself had been framed by Lawrence’s thoughts on Poe.

A fortnight earlier, in Colorado, I’d seen the Western Hostiles, which takes its epigraph from Lawrence’s essay on Fenimore Cooper: “The essential American soul is hard, isolate, stoic, and a killer.” Both experiences were shaped – one deliberately, by film-maker Scott Cooper, the other randomly – by what Lawrence had written.

****

There was a time when this was far from unusual. In 1945 Larkin had gushed to a friend that Lawrence was “the greatest writer of this century, and in many things the greatest writer of all times”. Born five years after Larkin, the narrator of John Fowles’s The Magus (1965) raises the bar higher as he leaves school believing Lawrence is “the greatest human being of the century”. And it wasn’t just the guys. Visiting the Lawrence ranch and shrine in Taos, New Mexico in 1939, WH Auden noted how “Cars of women pilgrims go up every day to stand reverently there and wonder what it would have been like to sleep with him.”

The answer, according to Kate Millett in Sexual Politics (1970), was probably unpleasant and certainly disappointing. Her devastating and witty analysis laid bare the trowelled-on sillinesses and “liturgical pomp” of exactly the thing Lawrence had become famous for, namely his writing about sex. Add in his regrettable – albeit temporary – infatuation with a proto-fascistic cult of “the leader-cum-follower” and it is easy to see why Lawrence’s reputation has been in more or less continuous decline since the 1970s. For a period that has now lasted as long as his short life (born in 1885, he died from tuberculosis aged 44 in 1930) Lawrence has managed to remain perversely out of step with whatever critical fashions have held sway. This is not to say that he has been without devotees. Hence the frustrated passion of Tony Hoagland, in his poem “Lawrence”, as he recalls how

On two occasions in the past twelve

months

I have failed, when someone at a party

spoke of him with a dismissive scorn,

to stand up for DH Lawrence

Any attempt at standing up for this “man who burned like an acetylene torch/from one end to the other of his life” had best begin by conceding not only the poor quality of novels such as Aaron’s Rod (1922) or The Plumed Serpent (1926) but that some of the canonical works are, in George Orwell’s understated words, “difficult to get through”. As a novelist, it could be argued, Lawrence peaked early, with Sons and Lovers (1913).

His former friend John Middleton Murry went further, arguing in a review of Women in Love (1920) that Lawrence was one of those novelists who “appear to have passed their prime long before reaching it”. Thereafter, as Raymond Williams has movingly expressed it, “what he lost along the way – what I think he knew he had lost and struggled to recover – may in fact be just as important as what he undoubtedly gained”.

One way to rebalance the books is to extend the critical catchment area to include forms of writings that are considered ancillary or minor. If Lawrence remains a great writer today that is due in no small part because his enduring freshness and force is found in the travel books, in poems that were scarcely even poems, and in the scatter of his essays. For Lawrence the novel, “the one bright book of life”, was the supreme test; that’s what he staked his life on. But many of his gifts were best displayed elsewhere. In this regard, in his inability to confine himself to the arena he most valued, he seems a distinctly contemporary writer: Lawrence as loose canon, so to speak.

An earlier compiler of a selection of Lawrence’s essays, Richard Aldington, decided that “Essays” was “rather a poor word for these brilliantly varied writings”. Actually, the brilliance and variety often occur within a single essay. (Dean Young, in his poem “Shield of Moon Dust”, narrows the focus still further, characterising Lawrence as someone “able to write horribly and magnificently/in the same sentence”.) Essays on writers are also essays on places; essays on places are also pieces of autobiography and so on.

Rebecca West, in the “Elegy” composed after Lawrence’s death, remembers meeting him in Italy where he was “tapping out an article on the state of Florence”. Later she realised that he’d actually been “writing about the state of his own soul at that moment”, using the city as a convenient symbol.

Persuasive and partly true, West’s analysis risks diminishing Lawrence’s uncanny ability to render what the American benefactor Mabel Dodge Sterne called “the feel and touch and smell of places” – which is why she invited him to New Mexico in 1921. On his circuitous way there from Sicily, in Australia the following May, he seemed immediately to intuit another world, some kind of dream-time: “a ‘fourth dimension’ and the white people swim like shadows over the surface of it”. While scarcely mentioning politics his “Letter from Germany” registers something – something “which has not yet eventuated” – blowing through the trees of the Black Forest in 1924: “Out of the very air comes a sense of danger, a queer bristling feeling of uncanny danger”.

Sensations flicker and blaze into ideas that are presented as though they are data from some instrument calibrated to a pitch of receptivity so extreme as to be abnormal or even pathological. The gulf between the ostensible object of enquiry and the direction taken by the investigation is frequently vast, the conclusions routinely drastic. Everything has the potential to become something else.

The best parts of “Art and Morality” (1925) are not about art or morality but – via an extraordinary speculative detour into the lives of ancient Egyptians – about how the “Kodak” habit of photographing oneself all the time has fundamentally changed our sense of ourselves: a prophetic diagnosis of a defining malaise of the iPhone era. In an editorial note to “Introduction to Pictures” the scholar James T Boulton rightly points out that the essay “does not once refer to pictures”. This tendency to stray from stated intentions was best expressed by Lawrence himself on 5 September 1914. “Out of sheer rage I’ve begun my book about Thomas Hardy. It will be about anything but Thomas Hardy I am afraid – queer stuff – but not bad.”

Intended as part of a series called “Writers of the Day”, the manuscript, which had veered far from any template, was not accepted for publication. Lawrence wanted it to leave the original brief still further behind and began recasting something that had been “mostly philosophicalish, slightly about Hardy” into a more explicit statement of his “‘philosophy’ (forgive the word)”.

Although Lawrence undoubtedly had a philosophy which he was keen to share with the world (to put it mildly), doing so involved him writing against his strengths. I love the way he took issue with Bertrand Russell for being unable to “accept in his philosophy the Infinite, the Boundless, the Eternal, as the real starting point” – but not withstanding his plans to “give some lectures on Eternity”, Lawrence was always at his best when facing the finite and the particular. However unpalatable they may be to modern readers, his “reflections” retain their fascination because of the way they are rooted not just in, for example, “the death of a porcupine” but in the agony of the dog with porcupine quills in his nose with which the piece begins. Lawrence was often carried away by stuff about a metaphorical “river of dissolution” but he noticed, with stunning clarity of vision, all the flora and fauna on the literal riverbank.

****

Balking instinctively at Joyce’s Ulysses (“Such effort! Such exertion!”), Lawrence was often best when most off-the cuff, even if that guaranteed Joyce’s reciprocally low opinion: “That man really writes very badly.” In this respect, although Lawrence re-wrote his major novels multiple times, the man who scorned Joyce for being “utterly without spontaneity” is like a proto-Kerouac. He could preach endlessly about man, woman and the need for them to be, as Birkin insists in Women in Love, “like two single equal stars balanced in conjunction”, but the best thing he ever wrote on the theme occurs as a throwaway line in the poem “For a Moment” when, sitting and waiting on a hotel terrace, he sees “the woman who looks for me in the world”.

No one has expressed more clearly than that woman, Frieda, what continues to engage readers who have otherwise grown weary of Lawrence’s loins of darkness, hard gem-like flames and so forth: “To me his relationship, his bond with everything in creation was so amazing, no preconceived ideas, just a meeting between him and a creature, a tree, a cloud, anything. I called it love, but it was something else – Bejahung in German, ‘saying yes’.”

There are moments like this in everything he wrote, irrespective of the form, from the description of cypress trees in Twilight in Italy (1916) – “For as we have candles to light the darkness of night, so the cypresses are candles to keep the darkness aflame in the full sunshine” – to a kangaroo with her “drooping Victorian shoulders” in the poem “Kangaroo”, to numerous scenes and passages in every one of the novels. You never know, in Lawrence, when or how the next flash of genius will manifest itself.

At times Lawrence’s writing floated free from the period of its composition, from the anxiously shared prerogatives of the age, in a way that rarely happens with the modernists who were his contemporaries. The opening of “Whistling of Birds” (1917) reads like a passage from JA Baker’s The Peregrine: “The frost held for many weeks, until the birds were dying rapidly. Everywhere in the fields and under the hedges lay the ragged remains of lapwings, starlings, thrushes, redwings, innumerable ragged, bloody cloaks of birds…”

Many of the later essays were done for money, when Lawrence lacked the energy and will to concentrate on sustained imaginative work. “I think perhaps it’s a waste to write any more novels,” he wrote to Nancy Pearn at the office of his literary agent Curtis Brown. “I could probably live by little things. I mean in magazines.” The patient Ms Pearn proved adept at finding homes for these pieces, which Lawrence considered “far the best way of making money”.

Irrespective of how such things originated, or of Lawrence’s motives in undertaking them, the act of writing invariably yields something – and this is the word with which the poet Tony Hoagland concludes his too-late defence – magnificent. Nowhere is this more evident than in the introduction he contributed to his friend Maurice Magnus’s posthumous Memoirs of the Foreign Legion. (1924). Largely about money, it was written to discharge Magnus’s debts and recoup money Lawrence had himself lost, but the extended length of the piece effectively wipes out – or writes off – its initiating motive. More generally, when Lawrence was dashing off “little things” he was in some ways playing to his strengths, unburdening himself, partly in the sense of expressing himself freely but also without the psychic weight of ambition that bears down on The Rainbow (1915) or Women in Love.

The carefree attitude he could bring to essay-writing had unexpected advantages. Throughout his career Lawrence threw out bottles containing messages energetically disdaining whatever it was he professed not to care about at any given moment. In one of the last pieces he wrote, an introduction to The Dragon of the Apocalypse by Frederick Carter – another example of a piece that outgrew its brief as Lawrence ended up writing his own Apocalypse – he set out some of the reasons why, and the ways in which, we continue to care about him.

I don’t care what a man sets out to prove, so long as he will interest me and carry me away. I don’t in the least care whether he [Carter] proves his point or not, so long as he has given me a real imaginative experience by the way, and not another set of bloated thought-forms.

He then goes on to anticipate our potential objections: “What does it matter if it is confused? What does it matter if it repeats itself? What does it matter if in parts it is not very interesting, when in other parts it is intensely so, when it suddenly opens doors and lets out the spirit into a new world, even if it is a very old world!”

On page after page of his essays Lawrence is revealed as a writer of endless renewal: our perpetual contemporary.

“Life with a Capital L: DH Lawrence Essays”, chosen and introduced by Geoff Dyer, is published by Penguin Classics on 31 January