‘If the fruits for life of the state of conversion are good, we ought to idealise and venerate it, even though it be a piece of natural psychology; if not, we ought to make short work with it, no matter what supernatural being may have infused it.’

From The Varieties of Religious Experience (1902) by William James

There is a long tradition of scientists and other intellectuals in the West being casually dismissive of people’s spiritual experiences. In 1766, the German philosopher Immanuel Kant declared that people who claim to see spirits, such as his contemporary, the Swedish scientist Emanuel Swedenborg, are mad. Kant, a believer in the immortality of the soul, did not draw on empirical or medical knowledge to make his case, and was not beyond employing a fart joke to get his derision across: ‘If a hypochondriac wind romps in the intestines it depends on the direction it takes; if it descends it becomes a f–––, if it ascends it becomes an apparition or sacred inspiration.’ Another ‘enlightened’ enemy of other-worldly visions was the chemist and devout Christian, Joseph Priestley. His own critique of spirit seership in 1791 did not advance scientific arguments either, but presented biblical ‘proof’ that the only legitimate afterlife was the bodily resurrection of the dead on Judgment Day.

However, there is good cause to question the overzealous pathologisation of spiritual sightings and ghostly visions. About a century after Kant and Priestley scoffed at such experiences, William James, the ‘father’ of American scientific psychology, participated in research on the first international census of hallucinations in ‘healthy’ people. The census was carried out in 1889-97 on behalf of the International Congress of Experimental Psychology, and drew on a sample of 17,000 men and women. This survey showed that hallucinations – including ghostly visions – were remarkably widespread, thus severely undermining contemporary medical views of their inherent pathology. But the project was unorthodox in yet another respect because it scrutinised claims of ‘veridical’ impressions – that is, cases where people reported seeing an apparition of a loved one suffering an accident or other crisis, which they had in fact undergone, but which the hallucinator couldn’t have known about through ‘normal’ means. The vicinity of such positive findings with ‘ghost stories’ was reason enough for most intellectuals not to touch the census report with a bargepole, and the pathological interpretation of hallucinations and visions continued to prevail until the late-20th century.

Things slowly began to change in about 1971, when the British Medical Journal published a study on ‘the hallucinations of widowhood’ by the Welsh physician W Dewi Rees. Of the 293 bereaved women and men in Rees’s sample, 46.7 per cent reported encounters with their deceased spouses. Most important, 69 per cent perceived these encounters as helpful, whereas only 6 per cent found them unsettling. Many of these experiences, which ranged from a sense of presence, to tactile, auditory and visual impressions indistinguishable from interactions with living persons, continued over years. Rees’s paper inspired a trickle of fresh studies that confirmed his initial findings – these ‘hallucinations’ don’t seem inherently pathological nor therapeutically undesirable. On the contrary, whatever their ultimate causes, they often appear to provide the bereaved with much-needed strength to carry on.

Rees’s study coincided with writings by a pioneer of the modern hospice movement, the Swiss-American psychiatrist Elisabeth Kübler-Ross, in which she emphasised the prevalence of comforting other-worldly visions reported by dying patients – an observation supported by later researchers. Indeed, a 2010 study in the Archives of Gerontology and Geriatrics addressed the need for special training for medical personnel regarding these experiences, and in recent years the academic literature on end-of-life care has recurrently examined the constructive functions of death-bed visions in helping the dying come to terms with impending death.

Kübler-Ross was also among the first psychiatrists to write about ‘near-death experiences’ (NDEs) reported by survivors of cardiac arrests and other close brushes with death. Certain elements have pervaded popular culture – impressions of leaving one’s body, passing through a tunnel or barrier, encounters with deceased loved ones, a light representing unconditional acceptance, insights of the interconnectedness of all living beings, and so on. Once you ignore the latest clickbait claiming that scientists studying NDEs have either ‘proven’ life after death or debunked the afterlife by reducing them to brain chemistry, you start to realise that there’s a considerable amount of rigorous research published in mainstream medical journals, whose consensus is in line with neither of these popular polarisations, but which shows the psychological import of the experiences.

For instance, although no two NDEs are identical, they usually have in common that they cause lasting and often dramatic personality changes. Regardless of the survivors’ pre-existing spiritual inclinations, they usually form the conviction that death is not the end. Understandably, this finding alone makes a lot of people rather nervous, as one might fear threats to the secular character of science, or even an abuse of NDE research in the service of fire-and-brimstone evangelism. But the specialist literature provides little justification for such worries. Other attested after-effects of NDEs include dramatic increases in empathy, altruism and environmental responsibility, as well as strongly reduced competitiveness and consumerism.

Virtually all elements of NDEs can also occur in psychedelic ‘mystical’ experiences induced by substances such as psilocybin and DMT. Trials at institutions such as Johns Hopkins University in Baltimore and Imperial College London have revealed that these experiences can occasion similar personality changes as NDEs, most notably a loss of fear of death and a newfound purpose in life. Psychedelic therapies are now becoming a serious contender in the treatment of severe conditions including addictions, post-traumatic stress disorder and treatment-resistant depressions.

This brings us back to James, whose arguments in The Varieties of Religious Experience for the pragmatic clinical and social value of such transformative episodes have been mostly ignored by the scientific and medical mainstream. If there really are concrete benefits of personality changes following ‘mystical’ experiences, this might justify a question that’s not usually raised: could it be harmful to follow blindly the standard narrative of Western modernity, according to which ‘materialism’ is not only the default metaphysics of science, but an obligatory philosophy of life demanded by centuries of supposedly linear progress based on allegedly impartial research?

Sure, the dangers of gullibility are evident enough in the tragedies caused by religious fanatics, medical quacks and ruthless politicians. And, granted, spiritual worldviews are not good for everybody. Faith in the ultimate benevolence of the cosmos will strike many as hopelessly irrational. Yet, a century on from James’s pragmatic philosophy and psychology of transformative experiences, it might be time to restore a balanced perspective, to acknowledge the damage that has been caused by stigma, misdiagnoses and mis- or overmedication of individuals reporting ‘weird’ experiences. One can be personally skeptical of the ultimate validity of mystical beliefs and leave properly theological questions strictly aside, yet still investigate the salutary and prophylactic potential of these phenomena.

By making this quasi-clinical proposal, I’m aware that I could be overstepping my boundaries as a historian of Western science studying the means by which transcendental positions have been rendered inherently ‘unscientific’ over time. However, questions of belief versus evidence are not the exclusive domain of scientific and historical research. In fact, orthodoxy is often crystallised collective bias starting on a subjective level, which, as James himself urged, is ‘a weakness of our nature from which we must free ourselves, if we can’. No matter if we are committed to scientific orthodoxy or to an open-minded perspective on ghostly visions and other unusual subjective experiences, both will require cultivating a relentless scrutiny of the concrete sources that nourish our most fundamental convictions – including the religious and scientific authorities on which they rest perhaps a little too willingly.