An old therapist of mine had a signed photograph of Sigmund Freud hanging on her wall. A gift from a former patient who had employed forgery skills in a side business of dubious legality, it was the iconic Freud photo: full suit, blank scowl, half-smoked cigar. Once, mid-session, I asked my therapist what she thought of Freud’s theories. ‘I don’t think much of them,’ she replied.

Her dismissiveness wasn’t a surprise. By any measure, Freud was one of the most influential thinkers of the 20th century. Following his death in 1939, the British author W H Auden was able to declare in his poem ‘In Memory of Sigmund Freud’ (1939) that Freud had represented ‘a whole climate of opinion’, and the subsequent two decades represented the heyday of psychoanalysis. No longer. Outside academia, for those who give it any thought, psychoanalysis is generally regarded as having followed phrenology and mesmerism into the dustbin of psychological enquiry. Boys lusting after their mothers; girls desiring a penis – such are the luridly risible impressions that persist in the popular imagination.

What went wrong? In 1996, Tom Wolfe wrote that ‘the demise of Freudianism can be summed up in a single word: lithium’. The American author described how in the early 1950s, after years of psychoanalytic ineffectiveness, rapid physical relief for sufferers of bipolar disorder arrived in the form of a pill. Wolfe’s example is microcosmic of a wider state of affairs. The waning of psychoanalysis corresponds precisely to the rise of modern neuroscience, whose physicalist approach now drives psychiatry. Today, almost anyone could have a go at describing serotonin, or dopamine, or Prozac. Few of those same people could define the primal scene, or the super-ego. As the American author Siri Hustvedt puts it in The Shaking Woman, or a History of My Nerves (2010), Freud is now seen by many if not most as ‘a mystic, a man whose ideas bear no relation to physical realities, a kind of monster of mirage who derailed modernity by feeding all kinds of nonsense to a gullible public until his thought was finally shattered by a new scientific psychiatry founded on the wonders of pharmacology’.

But in recent decades, this picture of philosophical antagonism has been complicated. Around 20 years ago, there emerged a new field, bearing the predictably cumbersome name of neuropsychoanalysis. Adherents to this amorphous research programme – spearheaded by the South African neuropsychologist and psychoanalyst Mark Solms of the University of Cape Town – are keen to rehabilitate Freud’s reputation for the age of the brain. They remind us that the young Freud started his career in neurology, and spent two decades in the hard sciences. They point to Freud’s attempts during the 1890s to ‘furnish a psychology that shall be a natural science’ and stress his lifelong belief that one day his theories would be augmented and refined by the empirical study of our grey matter. Neuropsychoanalysis published the inaugural issue of its academic journal in 1999, and held its first conference a year later. Since then, an increasing number of psychoanalysts have begun to investigate what neuroscience can offer their theories and practice, and conciliatory positions have been adopted by some of the most influential brain scientists of the era: Antonio Damasio, Joseph LeDoux, Jaak Panksepp, V S Ramachandran and, above all, Eric Kandel.

Could it be that Wolfe was wrong when he declared that the era of lithium means the end for Freud? Might the analyst’s couch and the brain scanner have something to offer one another?

Freud believed that, over the course of human history, humankind had suffered three ‘great outrages upon its naïve self-love’. First, there was Copernicus, who, with his finding that the Earth revolved around the Sun, showed that we were not at the centre of the Universe; second, there was Charles Darwin who, with his theory of evolution, showed that we emerged from the animal kingdom, and did not exist apart from it; last, there was Freud himself (he was never one for modesty), via whom psychoanalysis had shown that man was ‘not even master in his own house’ due to the massive effects of the unconscious. In a general way, neuroscience supports Freud’s idea of this third outrage. The idea of a vast and powerful unconscious is the central psychoanalytic concept that, it is claimed, the MRI scanner is now vindicating.

Kandel, a Nobel Prize-winning neuroscientist, is the most famous advocate of neuropsychoanalysis. In The Age of Insight (2012), he echoes Freud’s view that ‘most of our mental life, including most of our emotional life, is unconscious at any given moment’. And he points to two other big things Freud got right. First, ‘that the instincts for aggressive and for sexual strivings, like the instincts to eat and drink, are built into the human psyche, into our genome’. Second, that ‘normal mental life and mental illness form a continuum’.

‘Freudian slip!’ is rolled out as a humorous quip precisely because we don’t believe it was anything of the sort

There are other more specific sites of theoretical rapprochement, too – especially with regard to our understanding of memory. In the Mind Fields (2015), Casey Schwartz explores how modern research into memory reconsolidation suggests that long-term memories are revisable, supporting Freud’s claim that ‘memories are fundamentally dynamic’. The psychoanalytic concept of repression – in which shameful or traumatic thoughts are suppressed by the conscious mind to live out a troublesome existence in the subconscious – sounds fanciful. But it appears to have been at least partly supported by the brain sciences. When we are placed under extreme stress, the memory-forming hippocampus is bypassed, and experience registers in the fear centre of the amygdala, creating what LeDoux in ‘Psychoanalytic Theory: Clues from the Brain’ (1999) called an ‘unconscious memory’.

Equally, there are plenty of areas of Freudian theory that neuroscientists aren’t rushing to endorse. Freud got a lot wrong – ludicrously, colourfully wrong. Very few accept any component of his Oedipal complex, in which children unconsciously have sexual desires for their parents. No serious psychologist endorses his stages of psychosexual development, where over the course of their lives people progressively shift the focus of their libido from the oral through to the genital. And there are more fundamental issues. A central tenet of psychoanalysis is called psychic determinism – the idea that any and all mental and verbal processes, even those that might seem random or frivolous, in fact mean something.

Modern neuroscience, on the other hand, regards a great amount of thinking as ephemeral cognition and perceptual junk. (‘Freudian slip!’ tends to be rolled out as a humorous quip precisely because people don’t usually believe that what was just said was anything of the sort.) Similarly, Freud’s central idea that ‘every dream will reveal itself as a psychological structure, full of significance’ is hotly contested. J Allan Hobson, professor emeritus at Harvard, and others have spent decades arguing that dreams are in fact randomised imagery and confabulation, nothing to do with valuable unconscious secrets or dramatised wish-fulfilment.

One thing that makes Freud hard to assess in any sort of scientific way is that he was often most compelling as a philosopher, or cultural critic. In 1930 – a whole 15 years before humanity had witnessed the dreadful power of the atomic bomb – Freud wrote in Civilisation and its Discontents:

Human beings have made such strides in controlling the forces of nature that, with the help of these forces, they will have no difficulty in exterminating one another, down to the last man. They know this, and it is this knowledge that accounts for much of their present disquiet, unhappiness and anxiety.

This strikes me as a compelling idea. But we cannot test its veracity by examining blood flow in the brain. The truth is that you could spend all day enumerating what Freud might or might not have got right. The far more interesting thing is that, although neuropsychoanalysis always starts out with dry descriptions of conciliatory research, this isn’t really what the field is about. Its fundamental concerns run much deeper. In its debates, and its attempted or resisted rapprochements, neuropsychoanalysis dramatises a tension between two basic ways of thinking about what it means to be a human being: as a subject, and as an object. Or, to use an unfashionable dualism: as a mind, and as a brain.

How we think about mental functioning has always informed how we think about improving mental functioning. Neolithic man believed that mental distress was caused by evil spirits, and so tried to expel them by drilling holes in the skull. In the Middle Ages, the melancholic was adjudged in possession of an excess of black bile, so was sent for bloodletting, laxatives or maybe an exorcism. The same feedback loop between mapping the mind and healing the mind applies with psychoanalysis. When Freud had people lay on a couch in a dim room to talk about moments of childhood drama, he was both uncovering the structure of the mind, and searching for a way to soothe it.

Today, we live amid what some academics have termed a neuroculture – defined by a shift in both the popular and scientific consciousness towards an understanding of human life as being about what Fernando Vidal at the University of Barcelona called ‘being, rather than simply having, a brain’. As such, the modern psychological feedback loop is conspicuous for its materialism. When the depressed person is given SSRIs such as Prozac, it is because we believe that emotions are at bottom a physical phenomenon, thus requiring a physical intervention. It doesn’t matter so much what you, from within your skull, think about your situation. The path to change is all about how, from outside your skull, you can change the way the meat machine is running.

This grand ‘naturalistic turn in the human image’, as the philosopher Thomas Metzinger at the University of Mainz in Germany describes it, is at the root of everything from the slow death of religion to the rapid birth of the weirdly religious transhumanist movement. It is also the reason why psychoanalysis strikes many people as quaint, amateur, and probably impotent.

Psychoanalysis is built on the belief that introspection is a powerful force: that thinking has power

And it isn’t just about the prescription pad. The current era’s most prominent form of what used to be called ‘talk therapy’ – cognitive behavioural therapy (CBT) – is deeply un-Freudian. My old therapist, the one with the signed photo of Freud on her wall, was a specialist in CBT. She was a brilliant woman who relieved me of real pain. But my experience of CBT couldn’t have been further from some searing, quasi-mystical excavation of the shadowy underworlds of my psyche. In CBT, the substance and shape of painful states of mind are largely ignored, and a depressing or terrifying way of thinking about the world is simplified as something akin to a faulty algorithm, a poisonous song stuck in your head. It doesn’t really matter why you think you are in some elusive but elemental way empty, possibly better off dead. The point is to learn to analyse unhelpful patterns of thought, and correct them. If psychoanalysis is the Catholic confession booth, then CBT is more akin to the meditation cushion, or a chapter of Marcus Aurelius. The focus is on how one reacts to thoughts, not the deeper psychic significance those thoughts might carry.

As in psychiatry, so in everyday life. Introspection is out of fashion, and very rarely are modern conceptions of psychic self-improvement not based in an appeal to our physical, flesh-and-bone selves. We regard happiness, whatever that is, as basically a physical challenge. You might have noticed that people don’t seem to sunbathe or exercise these days; they ‘top up their vitamin D levels’ or ‘release some endorphins’. If you are a fan of meditation and mindfulness, are you not wooed by those articles about what happened when they put a monk in an fMRI machine? You might enjoy salmon, but don’t you also congratulate yourself for ingesting those omega-3 fatty acids? A friend told me recently that he had recovered from a bout of depression thanks to ‘paleo for the brain’: natural sleep patterns, long walks, lots of leafy greens. It isn’t Prozac, but the basic method remains the same: acting externally upon the physiology to improve wellbeing. In Aldous Huxley’s novel Brave New World (1932), the creation of an ‘ideal pleasure drug’ means no one ever has to experience unwanted emotions, living instead in a permanent state of glazed bliss. Huxley’s imagined world is a famous dystopia. But as a character in Michel Houellebecq’s novel Atomised (1998) remarks: ‘Everyone says Brave New World is supposed to be a totalitarian nightmare … but that’s hypocritical bullshit. Brave New World is our idea of heaven.’ In some sense, Brave New World is the perfect logical conclusion of the modern era, in which introspection is abandoned wholesale in exchange for perfectly calibrated physiological intervention.

By contrast, psychoanalysis is built on the foundational belief that subjective experience is primary, and that introspection is a powerful force in and of itself. That thinking has power. Thus, its therapeutic model is rooted in talking. For hours and hours, often for years and years. The basic position of psychoanalysis is a deeply empowering one: the mind has its own resources, and if you can learn to look inward with the right sort of gaze, you can begin to discern and to map your inner landscape. Perhaps not transformatively – Freud once said that the aim of his talking therapy was to ‘transform neurotic misery into common unhappiness’– but not insignificantly. As Hustvedt puts it, the only question one really needs to ask in evaluating psychoanalysis and its various descendants is: ‘Can talking free a person from symptoms?’ There is at bottom the belief that subjectivity alone can overhaul one’s internal landscape.

This basic tension between the brain sciences and psychoanalysis is the same tension at the heart of the so-called hard problem of consciousness: the seemingly irreconcilable clash between objective and subjective accounts of reality. A dedication to the value of the first-person viewpoint, an attempt to bring this into neuroscience, is what really lies at the core of the neuropsychoanalysis project. Neuropsychoanalysis, Solms explained to me, isn’t really interested in the long and labyrinthine history of psychoanalysis; it is interested in Freud’s original philosophical stance, which blended a respect for the natural sciences with a privileging of the individual mind.

‘Psychoanalysis is not important,’ Solms said, quoting what he often tells his students: ‘It’s about something important.’

Solms and others are passionate about brain research, but they are dismayed by the ‘eliminativism’ – the dismissal of the study of beliefs, desires and sensations – by thinkers such as Patricia Churchland at the University of California, San Diego and Daniel Dennett at Tufts University in Massachusetts. Neuropsychoanalysis is committed to the idea that ‘there are things that one can learn about the nature of the mental apparatus from this perspective, things that one can never see with one’s eyes, no matter how much you aid them with scientific instruments’, as Solms put it in a paper in 2011. Much of modern brain science, Solms told me, is embarrassed by the mind. Above all, neuropsychoanalysis wants to end the embarrassment.

‘We don’t need Freud; we need an approach which takes seriously the mental nature of the mind’

This is why Freud is less important to the field than what Freud represents. Researching this piece, I kept wondering: why hang on to Freud? He is an intensely polarising figure, so polarising that through the 1980s and ’90s there raged the so-called Freud Wars, fighting on one side of which were a whole team of authors driven (as the historian of science John Forrester put it in 1997) by the ‘heartfelt wish that Freud might never have been born or, failing to achieve that end, that all his works and influence be made as nothing’. Indeed, a basic inability to track down anyone with a dispassionate take on psychoanalysis was a frustration of researching this essay. The certitude that whatever I write here will enrage some readers hovers at the back of my mind as I think ahead to skimming the comments section. Preserve subjectivity, I thought, fine, I’m onboard. But why not eschew the heavily contested Freudianism for the psychotherapy of Irvin D Yalom, which takes an existentialist view of the basic challenges of life? Why not embrace Viktor Frankl’s logotherapy, which prioritises our fundamental desire to give life meaning, or the philosophical tradition of phenomenology, whose first principle is that subjectivity precedes all else?

Within neuropsychoanalysis, though, Freud symbolises the fact that, to quote the neuroscientist Ramachandran’s Phantoms in the Brain (1998), you can ‘look for laws of mental life in much the same way that a cardiologist might study the heart or an astronomer study planetary motion’. And on the clinical side, it is simply a fact that before Freud there was really no such thing as therapy, as we understand that word today. In Yalom’s novel When Nietzsche Wept (1992), Josef Breuer, Freud’s mentor, is at a loss for how to counsel the titular German philosopher out of his despair: ‘There is no medicine for despair, no doctor for the soul,’ he says. All Breuer can recommend are therapeutic spas, ‘or perhaps a talk with a priest’.

After Freud, though, there are doctors for the soul in abundance. And this doctoring starts with the individual; with his unique, inner view of existence. An adherence to Freud’s basic beliefs – that a science of subjectivity is possible, that such a science might help us to live – is what neuropsychoanalysis endorses by invoking his legacy. For Solms and others, however much Freud got wrong, the weight and fame and sincerity of what he was aiming for mean that he still deserves intellectual homage. I think you can make a decent case that preserving the Freud label is not worth the reputational cost. But as Solms himself put it to me: ‘We don’t need Freud; we need an approach which takes seriously the mental nature of the mind. And since Freud was the one who has done that most thoroughly, that seems to me the correct place to start.’

Recovering or conserving the subjective viewpoint is an appealing notion, and its appeal reveals the sort of doublethink we have towards our era of the brain and the fMRI. Everyone’s attention is grabbed by talk of endorphins and serotonin-boosters and cooling that pesky fight-or-flight response. However, even when the evidence suggests that we want the total physiological control of Brave New World, it remains one of the most famous literary dystopias of our time. There is still horror in the idea of totally relegating our individual experience to the species-wide logic of physiology. Most of us, deep down, want our inner lives – our ideas about ourselves, our sense of where we want life to lead, what we fear, what we desire – to be of consequence. Psychoanalysis is attractive at least partly because it makes us rich, narrativised, and mysterious to ourselves. It makes life a novel, not a textbook.

Certainly, the project can become overheated, indulgent: to imagine that the depths of our being resembles the timeless turbulence of a Greek myth flatters us. To imagine that our dreams are loaded with meaning plays to our basic narcissism. (The same narcissism that makes us want to tell people all about them, and makes those that aren’t ours so reliably dull.) But there is a basic principle at play here, and it has to do with the feeling that no generalised theory can capture even a single living mind, let alone all of them. Rationalising ourselves offers a form of relief. No more vagueness; everything measurable and editable.

And yet, on some level, we don’t want to live our lives purely according to biochemistry, the same biochemistry as 7.5 billion others. What a flattening! In such an equation, something gets lost, even if we struggle to say what. Not for nothing does psychoanalysis survive most unscathed in the humanities. Freud’s work references Hamlet and Macbeth, and Goethe’s Faust. For a century, writers including H G Wells, Virginia Woolf, J G Ballard and Paul Auster have been compelled by his work. Like psychoanalysis, the humanities (and especially literature) privilege the richness of the individual life, and regard reality as populated by subjects rather than objects. Like psychoanalysis, the humanities are often framed as in decline, dwarfed by the technocratic bloodlessness of a scientistic age. There is a parallel pursuit for the two projects. Both are driven by the same instinct, that the stories we tell ourselves can affect how we live with ourselves.

It’s fashionable and liberating to talk about there being no essential self – but it’s a damn hard way to live

In this contradictory context, neuropsychoanalysis presses on, trying to supplement the study of the brain with the experience of what it is like to emerge from one. To have the neural explanations do justice to what the novelist Vladimir Nabokov called ‘the marvel of consciousness – that sudden window swinging open on a sunlit landscape amidst the night of non-being’.

The strained project reveals us to ourselves. In Illness as Metaphor (1977), Susan Sontag wrote that ‘a large part of the popularity and persuasiveness of psychology comes from its being a sublimated spiritualism: a secular, ostensibly scientific way of affirming the primacy of “spirit” over matter’. Freud was a committed atheist who rolled his eyes at what he called the oceanic feeling. But today, to believe in a transformative role for introspection is to ally oneself to conceptions of individual agency more commonly invoked by spiritual traditions than by modern-day science. And spirituality in all its mutant forms won’t quite die for the same reason that psychoanalysis won’t quite die.

It’s fashionable to talk about there being no essential self, and it’s a tantalisingly liberating idea, in its way – but it’s a damn hard way to live. From within the skull, it feels like our view of the long walk from cradle to grave matters, matters an awful lot – and that the kaleidoscope of experience coalesces continually into a strange bright point still deserving of that outmoded label: soul. I have to agree with Solms; it can feel like a great slow-motion tragedy, this relegating of the mind. Friedrich Nietzsche believed that we hadn’t arrived at a real atheism yet, and had just hoisted humankind onto a pedestal that was secretly divine. Can you blame us? Human exceptionalism is our master religion. If it is indeed ourselves that we now worship, then in a sense Sontag’s sublimation is revealed as a sort of double bluff, a way to render matter as primary, yet still imbued with something else.

The neurosciences are a miracle of science. We need them, and we are right to be enthralled by them. I too live in the doublethink; I too chase the vitamin D and the omega-3s. But there remains an allure to the Freudian project, however initially misguided. I recall many years ago spending 10 minutes with an overworked GP before being shuffled off with a Zoloft prescription. Around the same time, I discovered the work of Albert Camus and felt its acceptance of the bittersweet majesty of life like the hand of God on my shoulder. I never did take the Zoloft, and came to find that new ways of seeing the world were enough, more than enough. The most meaningful discoveries of my inner life – Fyodor Dostoyevsky, George Orwell, Buddhism, the band Tool – have penetrated my consciousness in the form of what feels like pure thought, thought that I have absorbed and projected back on to reality as an actor, a being, a witness. How this side of life could possibly be folded into a neuroscientific framework, I don’t know. But it feels like, Freud or no Freud, we should try.