I was in my late thirties before it struck me that there was something odd about the tableau I have in my mind of a familiar living-room, armchair, my father in it, silvery hair, moustache, brown suede lace-ups, and me, aged six or so, sitting on his knee. The layout is correct – I have been back to the block of flats and sat in the living-room of the flat next door, with the same floor plan. Door in the right place; chair I’m sure accurate, a burgundy moquette; patterned carpet; windows looking out onto the brick wall of the offices opposite. My father looks like my father in pictures I have of him. I look like … well, actually I don’t have any pictures of me at that age. But I’m sure I looked pretty much like the memory I can call up at will. It’s not particularly interesting as a memory. Nothing special is happening. It could be a painting, or a photograph, except that I shift about as a child does sitting on her father’s knee. Here’s the thing, though: I can see the entire picture. I can, you may have noticed, see myself. My observation point is from the top of the wall opposite where we are sitting, just below the ceiling, looking down across the room towards me and my father in the chair. I can see me clearly, but what I can’t do is position myself on my father’s knee and become a part of the picture, even though I am in it. I can’t in other words look out at the room from my place on the chair. How can that be a memory? And if it isn’t, what is it? When I think about my childhood, that is invariably one of the first ‘memories’ to spring up, ready and waiting: an untraumatic, slightly-moving picture. It never crossed my mind to notice the anomalous point of view until I was middle-aged. Before then it went without saying that it was a ‘real’ memory. Afterwards, it became an indicator of how false recollection can be.

Memory has always been a worry to us. The thing we feel sure makes us ourselves (no memory, no me) is also something we know to be treacherous, overaccommodating, fugitive: delightfully and fearfully unreliable. We’re stuck inside our own heads with our recollections (or old photos and now videos that have become memories) and there is no way, except sometimes by trusting to the probably unreliable memories of other people, to be absolutely sure that we know what we think we know, or are who we think we are. That anxiety about the accuracy of our grasp of our past selves accounts for the way many other alarming aspects of being alive have become attached to the subject of memory; the theme changes and goes through cycles over time (law, war, politics, medicine, family, sexuality), but always serves to remind us to worry about the consequences of never being quite sure of what we and others remember. People have thrown all the expertise they can find or invent at the problem. We have asked shamans, clairvoyants, hypnotists, historians, scientists, surgeons, law-makers, artists and writers, social psychologists and psychoanalysts to investigate the truth, the facts, the interpretations, so to reassure us about the mechanism and reliability of remembering, but, as Alison Winter’s deft study of 20th-century memory controversies concludes, we haven’t come close to a definitive answer.

Yet, alongside our anxiety about the trustworthiness of remembering, there is an opposite pull, which is quite as powerful, towards the commonsense feeling that we can all know and trust our own memories; that we know our own minds. Memories when they rise feel reliable. Whatever scientists or other experts do in the laboratory, library or consulting room, individuals, including the experts themselves when off duty, proceed in their everyday lives as if their personal memories are a valid basis for action and interaction, just as physicists continue to walk on apparently solid floors while knowing that they are largely made up of empty space. We would be mad not to. Underlying the compelling feeling that we are our memories is a further common-sense assumption that our entire lives are accurately retained somewhere in the brain ‘bank’ as laid-down memories of our experience, and that we retrieve our lives and selves from an ever expanding stockpile of recollections. Or we can’t, and then that feeling that it’s on the tip of our tongue, or there but just out of range, still encourages us to think that everything we have known or done is in us somewhere, if only our digging equipment were sharper. It’s considered a fault not with recording, but with playback. I was in no doubt about that as a small child. I had a small deep-red memory stone lodged in my left temple, and when I was asked a question at school it moved slowly and steadily from one side of my forehead around to the other. Before it was at the midway point, I tried for the answer, knowing it was in my mind, available to me; but once the stone passed the centre line between my eyes, I stopped worrying about it: I knew I didn’t know the answer, it simply wasn’t ‘there’. I supposed it was how everyone knew what they did and didn’t know. Looking back, it was an efficient filing and retrieval machine that unhandily had vanished by the time I reached secondary school. I recall the memory stone with some nostalgia; these days it’s the inefficiency of my mind-machine that exercises me.

It’s very difficult not to see the depleting memory of ordinary ageing as making you somewhat less of the person you were. Every time someone calls me back in a shop because I’ve left my purse or forgotten to pay, or I fail to remember that whatsisname was married to Carole Lombard and, increasingly, that whatsisname – oh you know, the bloke who played uh … Rhett Butler … was married to whatsername … begins with an L or an H … it feels to me that I am not who I used to be, not quite myself, or that I am continuing to leave ever further behind the someone I was. It isn’t the information that Carole Lombard was married to Clark Gable that has gone, it’s the me who knew it who is disappearing. Those who are older than they are young make exaggeratedly impatient, self-deprecating jokes when they forget a name, a face or why it was they walked into a room. (Recent research from Notre Dame suggests that it may be passing through doorways which unframes the thought you had the second before – but I’ve just forgotten the end of this sentence and I haven’t moved, let alone left the room.) That ubiquity of joking, nervous laughter as we confess to a memory lapse suggests we know very well that the increasing frequency of the loss of a recollection means much more than an irritating moment of blankness. It’s the ‘normal’ beginning of the loss of ourselves, and it is terrifying. Beneath the laughter, blind panic.

Our two conflicting beliefs – that memory is infallibly impressed in the library of our brains, and that memory is unreliably subject to physical and cultural influences – are the alternating strands that structure Winter’s book on the past century of memory research. Popular understanding of how our minds work has been mirrored by two theories which vie with each other for dominance. Memory is authentic, a perfectly preserved neuronal record of every experience retained in the brain; or it is reconstructive, a dynamic mixture of recorded past experience continually reassembled according to past events and present information, and conventionalised to make sense of current cultural and personal contexts and cognitive norms. In the 1950s, the neurosurgeon Wilder Penfield, operating on epileptics in Montreal, proved to his satisfaction that memories were individual dormant records, faithful in the manner of filmstrips of serial events, each of which could be brought into consciousness by stimulating the same particular areas of the brain. Patients operated on by Penfield remained awake and able to recollect entire songs (not snatches, but chorus and verse), or relive incidents from their past lives, depending on the precise part of the temporal cortex Penfield triggered with his electrode. He could return to the same group of neurones and evoke the same song, the same experience over and over. What’s more, there was a measurable time lag between the firing of the stimulated neurones and the memory’s coming to consciousness: the librarian retrieves the book sitting there in the stacks and sends it out at the reader’s request.

In 1913, at the Psychological Laboratory in Cambridge, Frederic Bartlett was of the opposite opinion, believing memory to be a schema, seeded by experience but fleshed out by a plethora of social, psychological and cultural circumstance. He devised tests, asking members of the public to observe and then recall certain geometric shapes, pictures and faces. He found that people gave both abstract and representational stimuli names, meanings and narratives according to personal and social needs, desires and understanding, and that these conditioned what and how they remembered what they’d been shown. He concluded that we remake our memories each time we think them. The opposing theories of Penfield and Bartlett reflected positions held long before they did their work, and are still referred to by contemporary neuroscientists.

Fiction takes a non-specialist and usefully cavalier interest in the more intriguing arguments of science, and the dangers of remembering and forgetting are staples of science fiction. The identity terrors in much of Philip K. Dick’s work and the movies based on it (Do Androids Dream of Electric Sheep? – Blade Runner; We Can Remember It for You Wholesale – Total Recall), and Pope’s ‘eternal sunshine of the spotless mind’ revisioned by Charlie Kaufman for the cinema, celebrate the humanity of human beings as the accumulation of their experience. In Dick the androids are given a lifetime of false memories to persuade them they are human, while Jim Carrey’s mind in Eternal Sunshine eventually revolts at excising the emotions associated with a lost and painful love affair. We like to think we are our pain and happiness, that experience and the recollection of it as event and emotion, our possession of it, builds into our moral character. Winter quotes from the President’s Council on Bioethics, convened by George W. Bush in his first term, which considered, alongside stem cell research, the new possibilities of memory erasure.

Editing memories could ‘disconnect people from reality or their true selves’, the council warned. While it did not give a definition of ‘selfhood’, it did give examples of how such techniques could warp us by ‘falsifying our perception and understanding of the world’. The potential technique ‘risks making shameful acts seem less shameful, or terrible acts less terrible, than they really are’.

The law has fixated on memory as much as science and fiction have. Witness evidence is at the heart of trial by jury. But even supposing people want to tell the truth, how can we trust their recollections, and following on from that, whose expertise about the truth or fallibility of memory is reliable? Many popular crime and legal dramas depict the procedures for finding out what the suspects won’t reveal, or for getting them to confess. But more disturbing for the notion of justice is the possibility that people don’t know the truth: that witnesses, victims and even perpetrators misremember, or remember according to criteria unconnected with what actually happened. Winter begins her book with the story of a young man, Richard Ivens, who in 1906, as a result of intense interrogation and possibly hypnosis, came to believe – that is, to remember – that he had murdered a woman whose body he had found, a crime he later denied but was executed for. His case was taken up by two Harvard psychologists, Hugo Münsterberg and William James, who were disturbed not just by the possibility that Ivens’s confession was false, but about the mental process that resulted in his seeming genuinely to remember in detail carrying out the killing. Legal systems depend on someone knowing all or part of the truth, and someone else judging whether the truth is being told. Witnesses to crimes regularly misremember, and even at the beginning of the last century, those notable psychologists believed that the accused, too, could be manipulated into ‘remembering’ wildly. Lay juries listen to testimony, but in reality find themselves having to judge between experts offering differing opinions on the likely truth of the evidence given.

In the middle of the 20th century science busied itself devising techniques for retrieving the truth that was supposed to be embedded in minds unable or unwilling to access it. It wasn’t just the law that would supposedly benefit from the magic elixir known as ‘truth serum’ – scopolamine hydrobromide, sodium amytal and sodium pentothal. The military attempted to dig secrets out of enemy prisoners, and psychotherapists, wanting to ease the agony of the war-traumatised or render them fit to return to military service, fell on the drugs and combined them with techniques of brainwashing or abreaction. A drugged twilight sleep was believed to be an innocuous third degree guaranteed to release suppressed memories that the subject would be incapable of controlling. You got the whole truth from an enemy or a criminal, while the mentally wounded relived their terrible experiences, no longer dissociating, and as a result apparently felt altogether better. I suppose if such a serum really worked – if some substance made people retrieve, speak and relive super-accurate memories and experiences – the world would be a paradise of peace and harmony. Unless it turned out to be such a hellhole that drug companies would be scrambling to find an untruth serum to get us back to the bliss of lies and forgetfulness.

One compelling basis for the idea that the remembered truth would cure you was psychoanalysis, which saw the recovery of suppressed experience as the royal road to maturity. It too assumed that everything was available but that what the unconscious did in suppressing the unpleasant was not, after all, in the organism’s best interest. It was a well-meaning but inefficient protection system that came back to bite you some way or other. Releasing the bad ‘forgotten’ memories and confronting them, Freud believed, would free us from a life of repeating the consequences of early trauma and allow us to lead a truly considered, independent existence. In the 1980s and 1990s this attractive proposition led to ugly consequences. The assumption that we possess but repress memories too difficult to bear, and that it would benefit us to confront them, was combined with confusion about Freud’s theories concerning his patients’ real or fantasised incest experiences, with the result that an excited group of psychologists and social workers went on a crusade to release women from the horrors of their almost inevitably uncovered childhood sexual abuse. The ease with which ‘recovered memories’ of being raped by fathers, or subjected to satanic abuse by parents and neighbours, were ready to emerge from their hiding place in the unconscious as a result of little more than enthusiastic suggestion and a bit of sympathy, makes one wonder that anyone retained any belief at all in the power of the repressive unconscious. That incest occurs, and probably more often than we care to imagine, is not in doubt, but during this period so many women dredged up so many memories of sexual abuse that after criminal charges and civil litigation brought by both parents and children against each other peaked in the 1990s, the American Medical Association finally declared, as Winter reports, ‘that recovered memories were “of uncertain authenticity” and needed “external verification”’. The American Psychiatric Association warned that it was ‘impossible to distinguish false memories from true ones’.

That recollection I have of myself as a child shifting about on my father’s knee has no context: it would be a perfect vehicle for a psychotherapist with a commitment to the ubiquity of childhood sexual abuse to suggest that it is a screen memory. Even as I described the scene, I imagined the reader’s mind moving tentatively towards a revelation. I felt the weight of expectation myself, as I wrote ‘shift about as a child does sitting on her father’s knee’. Surely, if I had been writing a novel or an autobiography, it would be the prologue to a recollection of sexual abuse. The recovered memory movement may have been replaced with the idea of false memory syndrome, but it has left its cultural mark. Nevertheless, the debate between recovered and false memory recycles the same old argument: either we remember everything accurately and it can be retrieved, or our so-called memories are highly plastic and we are inclined to remember according to our own and others’ expectations.

Winter’s final, disturbing chapter discusses how work on the neurophysiology of remembering is shaking the most basic assumptions we hold about memory, trauma and the danger of repression. Eternal Sunshine of the Spotless Mind took its premise of memory erasure not from science fiction, but from contemporary brain research. It seems that the emotion of an event is stored separately (in the amygdala) from the recollection of the event itself (in the hippocampus), and that traumatic memories are physiologically different from the regular sort. A traumatic experience is accompanied by a surge of adrenal stress hormones which increases the strength of the memory, and each time the event is recalled, a renewed rush of epinephrine and cortisol reinforces the event’s emotional impact and its ease of recall. In other words, each time you remember something awful, the memory and its associated distress are strengthened. The trauma is re-created and enhanced with every recollection. At NYU and the University of California at Irvine they’ve tested rats and people by creating emotionally disturbing or distressing events and then injecting the amygdala of some of the subjects with protein-inhibiting chemicals. Those people and animals were able to remember or relive what they experienced without any emotional distress, and they were more easily able to forget the event, while for the untreated subjects the distress became measurably more vivid each time.

If these studies are to be believed, confronting or working through disturbing, repressed memories, or post-traumatic stress, would continually reinforce the damage, while not thinking about it actually weakens the unbearable memory. Eternal sunshine requires a spotty, not a spotless mind. Keep the good stuff, snuff out the bad. So where the Victorians believed in hard experience building character, and the 20th century put its faith in facing demons, we may be looking at a new future of positive forgetting. Army veterans in the US have been prescribed adrenal hormone-blocking propranolol for PTSD, and Winter suggests the possibility that we might be responding negatively to memory editing in much the same way as people objected to pain-killing and anaesthetics when they were first discovered. My almost automatic resistance to the notion of drug-induced benevolent forgetting is perhaps simply proof of the influence of my time and culture. We might need to fear our memories more and develop a new appreciation of our repressive unconscious.