I love William James. He’s just about the only philosopher who didn’t end up as either a pettifogging nit-picker or an overbearing egomaniac with delusions of genius. He was generous too – witty, honest, modest and flexible – and more interested in promoting productive conversations than hogging the last word. He was also a brilliant writer. At first glance, his prose may look like an easy outpouring of spontaneous colloquialisms, but in fact he took great pains to make it cover lots of rough ground without any hard words and without any tired ones either. He was a brilliant phrase-maker too: inventor of “subliminal consciousness”, the “divided self” and the “sick soul”, of “mental states”, the “stream of consciousness” and, last but not least, “religious experience”.

James was not unsympathetic to religion, and on occasion he was prepared to call himself a Christian, though in a thoroughly secular and untheological sense. His abiding intellectual passion was a love of open-mindedness and a corresponding distrust of dogmatism and metaphysics. We should never forget, he said, that all our opinions – even our “most assured conclusions” – are “liable to modification in the course of future experience”. But he warned against allowing a distrust of dogmatic metaphysics to harden into a metaphysical dogma of its own, as seemed to be happening with some of the evangelising atheists of his day. He admired the evolutionary biologist T H Huxley and the mathematician C K Clifford, for example, but when they used the idea of “science” as a stick to beat religion with they were in danger of behaving like high priests of a new religion – “the religion of scientificism” – and defending it with the same intolerant zealotry as any old-style religious fanatic. Knowledge, for James, was not so much the pre-existing premise of human inquiry as a hoped-for future product, and science was more like a tissue of fortuitous insights than a monolith of solid fact. We would not have much chance of stumbling into truth if we let ourselves get too anxious about falling into error, and the first rule of an unillusioned epistemology should simply be: Relax! “Our errors are surely not such awfully solemn things,” James wrote: “in a world where we are certain to incur them in spite of all our caution, a certain lightness of heart seems healthier than this excessive nervousness.”

James toyed with various titles for his light-hearted approach to truth: “pragmatism” was one, though that made it sound too like Nietzschean immoralism; “pluralism” was another, except that it had connotations of neo-Kantian relativism; so perhaps the best option was “radical empiricism”, which was also a way of paying tribute to the paragon of intellectual modesty and progressive open-mindedness, the still lamented John Stuart Mill. There was one further label that James was tempted by – suggested to him by his English disciple F C S Schiller, a gifted but unclubbable Oxford don. “I have been inspired with THE name for the only true philosophy,” Schiller told him in 1903. “Why should we not call it Humanism? Humanism as opposed to Scholasticism, ‘humane’ as opposed to barbarous….” James was intrigued but not quite convinced: “‘Humanism’ doesn’t make a very electrical connection with my nature,” he told Schiller, though he was happy to let him do what he could with it. And if, as some of us think, the word “humanist” is now a term in search of a meaning, perhaps it is time for it to meet up again with the philosophy of William James.

James was born in New York in 1842, and brought up to be a creative free spirit. His father, Henry James Sr, was wealthy and cultivated, an amateur writer and lecturer with an interest in philosophy, art, unorthodox religion and progressive politics, and he ran through most of his wealth taking his wife and their five children – William, Henry, Wilkie, Bob and Alice – on protracted cultural tours of Europe, where they could imbibe the great languages and literatures at source, become connoisseurs of the finest art and music, and thus grow up to be emblematic ambassadors of the bountiful socialistic freedom that awaited the human race.

The education of the James siblings, though enviable in many ways, left them ill-equipped for careers back in America. They spent their early lives as “hotel children”, as Henry Jr. would recall, and they ended up with plenty of culture but not much education, their formal training being confined to “a few small vague spasms of school”. At the age of 18 William decided to become a professional artist, and spent a year in a painter’s studio in Newport, Rhode Island, before changing his mind and taking up applied science at Harvard. Henry joined him there to study law, though it bored him and he spent most of his time reading crime fiction. In the meantime the United States had embroiled itself in civil war, and whilst William and Henry were excused military service on account of ill health (bad eyes and bad back respectively), Wilkie and Bob led companies of black soldiers into battle in the Union cause, suffering physical and mental wounds from which they would never recover, while worry over them turned Alice into an invalid for life.

In 1865, when the war was over, William spent the best part of a year on a scientific expedition to Brazil, hoping to train himself as a practical field naturalist, though when he returned to Harvard he decided to undertake a medical training instead. Meanwhile Henry was beginning to earn large sums by selling stories to prestigious American magazines, and William, disheartened at his brother’s success, sank into prolonged depression. It was not till 1873, when he was already in his thirties, that he won himself a measure of independence and self-respect by taking a post – though a very lowly one – as a teacher of physiology at Harvard.

James soon became a committed Darwinist, laying special stress on the doctrine that natural selection operates on variations that are, from a biological point of view, random and haphazard, and hence (contrary to what many self-styled evolutionists said) that the development of species was essentially open-ended and unpredictable. He also shifted his area of teaching from the body to the mind, and undertook to write an introductory textbook on psychology. It took him longer than he expected – the manuscript was delivered to the publisher 12 years late – but it was well received and is still widely regarded as the founding classic of the discipline.

Principles of Psychology, published in 1890, was one of the first works to take up and generalise the idea of a secondary or “sub-conscious” self, first proposed by the French neurologist Jean-Martin Charcot. Apart from his early advocacy of a notion later appropriated by Sigmund Freud, James’s principal objective was to discredit what he called “psychological atomism” – the idea that experience consists of a succession of distinct ideas and that perception, intellect, emotion and will are separate mental faculties, complete and entire in themselves. For James, there could be no such thing as a permanent, substantial anchor for our personal identity – no “self,” but only the changing kaleidoscope of “what we back ourselves to be and do”, and the mind as a whole was simply a “theatre of simultaneous possibilities” or a “field” of fluid forces animated by our bodily engagement with the world.

The arguments of the Principles were as much epistemological as empirical, and James issued frequent warnings against what might be called perfectionism or completism in the field of theory. He acknowledged, in particular, that the relations between mental states and the brain were deeply problematic; but insisted that if psychology was to make progress as a science it must simply look the difficulty in the eye and pass on. Once the Principles were published James began to explore these kinds of issues in a series of “Essays in Popular Philosophy”. Scientists, he suggested, had more in common with artists than we usually think, and they would not have much chance of making interesting discoveries unless they were prepared to back their hunches and take cognitive risks. Knowledge had to be led by belief, not the other way round; and if we wanted to acquire it we had a right, even a duty, to believe.

The doctrine that belief had priority ever knowledge came to public notice in the title essay of James’s first philosophical book, The Will to Believe (1897), and readers were quick to recognise that it promised to throw new light on the most agonising intellectual problem of the nineteenth century: the question of what would remain of religion if primitive superstition were discarded. So when a committee in Scotland found itself having to choose someone to give the next set of Gifford Lectures in Edinburgh, they naturally thought of James. They would have preferred someone more “orthodox”, but they were running out of options and their brief was very accommodating – the lectures had been set up ten years before to promote a modern, liberal approach to religious affairs, and lecturers were permitted to be “of any religion, or of none”. James was therefore invited and quickly agreed to come, offering a two-year course on “Religious Experience”, starting in 1899. Then he lost his nerve, fell ill, and tried to pull out, but they held him to his promise and in the spring of 1901 he came to Edinburgh and started delivering the lectures that would become his most celebrated book: The Varieties of Religious Experience.

He was very anxious. His brother Henry had by then established himself as an honorary English gentleman and one of the finest novelists of all time; but William was no more than an empirical scientist, a convinced Darwinian and an unabashed colonial hick – he dressed and spoke and thought like a Yankee, and admitted that he had been won over by the size of the Gifford fee, which would help finance his habit of “roughing it” on his farm at Chocorua in the wilds of New Hampshire.

The title of the first lecture was “Is Religion a Nervous Illness?” – and there was general relief when James concluded that it was not. He went on to build an impressive case against “survivalism”, or the idea that modern religion could be dismissed as an etiolated relic of deluded prehistoric prejudices. (He pointed out that the same could be said with equal truth of the presuppositions of modern science.) He went on to criticise the survivalists for “intellectualism” – in other words, for setting too much store by theory, and failing to see that “feeling is the deeper source of religion, and philosophic and theological formulas are secondary products, like translations of a text into another tongue.”

The feeling most fundamental to religion was a sense of reverent wonder in the midst of the world as we find it, and its defining characteristic was, James said, “a willingness to close our mouths and be as nothing.” The opposite of religion was not so much atheism as the adoption of a “trifling, sneering attitude” towards everything, after the manner of the 18th-century French cynic and tedious, tinder-dry chatterbox Voltaire. Religion meant being able to turn your back both on ironic frivolity and on ponderous complaint: “it favors gravity, not pertness,” James wrote, and “says ‘hush’ to all vain chatter and smart wit.”

From a purely psychological point of view it was obvious that religion – with or without a belief in God – did people good: it made them less self-centred, more willing to forgo personal advantage, and better able to live a simple life and find satisfaction in friendship, devotion, trust, bravery and patience. But it also gave life a fresh charm – “an enchantment which is not rationally or logically deducible from anything else.” And this enchantment could furnish cognitive rewards as well: religion transports us to new vantage points and gives us access to truths that were otherwise sure to elude us. “Why should we suppose that only one system of ideas can be true?” he asked. “Why may not the world … consist of many realities, which we can approach using different conceptions, just as a mathematician might use geometry, algebra or calculus?”

James spent the next eight years trying to promote his pragmatic, pluralist, empiricist approach to truth – what some would call his humanism. He made it clear that his sympathy for religion did not extend to “religion’s wicked partner – the spirit of dogmatic dominion, the passion for laying down the law”. He favoured philosophy too, but not the dreary timidity encouraged by what he called “the PhD Octopus”, nor the stupefying smugness and philistine cleverness propagated by a posh young Englishman called Bertrand Russell. “The wisest of critics is an altering being,” he wrote, always “subject to the better insight of the morrow.” But while his studious anti-dogmatism found a hearing all round the world, he was unable to nurture it and make it take root before death stopped him in his tracks, on 26 August 1910, at the age of 68.

He was not afraid of dying, but he was annoyed to be doing it so soon. “It has come so rapidly, rapidly,” he said, as he lay in bed at Chocorua, dosed with morphine and barely able to breathe. But if his works have not been as widely read as they deserve, they have been able, from time to time, to strike lights in the lives of others – most notably Ludwig Wittgenstein, who of all the great philosophers was always the hardest to please. Bertrand Russell allowed himself to be thoroughly vexed in 1912, when Wittgenstein kept praising Varieties of Religious Experience and telling him that “it does me a lot of good.” Towards the end of his life Wittgenstein told his old friend Maurice Drury – in words that, coming from him, were far from trite: “that is what made William James such a good philosopher: he was a good human being.”

[Photos from top: William James; James and Alice, his wife; James and his brother Henry; James and his daughter, Peggy. All from the excellent William James Archive at Emory University]

More heroes of humanism: Read Jonathan Rée on JS Mill, Jürgen Habermas, Leszek Kowlakowski, Hugh Trevor Roper, Isaiah Berlin and Daniel Dennett.