When Rachel Benmayor was admitted to hospital, eight and a half months pregnant, in 1990, her blood pressure had been alarmingly high and her doctor had told her to stay in bed and get as much rest as possible before the baby came. But her blood pressure kept rising – this condition, known as pre-eclampsia, is not uncommon but can lead to sometimes-fatal complications – and the doctors decided to induce the birth. When her cervix failed to dilate properly after 17 hours of labour, they decided instead to deliver the child by caesarean section under general anaesthetic. Rachel remembers being wheeled into the operating theatre. She remembers the mask, the gas. But then, as the surgeon made the first incision, she woke up.

“I remember going on to the operating table,” she told me. “I remember an injection in my arm, and I remember the gas going over, and Glenn, my partner, and Sue, my midwife, standing beside me. And then I blacked out. And then the first thing I can remember is being conscious, basically, of pain. And being conscious of a sound that was loud and then echoed away. A rhythmical sound, almost like a ticking, or a tapping. And pain. I remember feeling a most incredible pressure on my belly, as though a truck was driving back and forth, back and forth across it.”

A few months after the operation, someone explained to Rachel that when you open up the abdominal cavity, the air rushing on to the unprotected internal organs gives rise to a feeling of great pressure. But in that moment, lying there in surgery, she still had no idea what was happening. She thought she had been in a car accident. “All I knew was that I could hear things … and that I could feel the most terrible pain. I didn’t know where I was. I didn’t know I was having an operation. I was just conscious of the pain.”

Every day, specialist doctors known as anaesthetists (or, in the US, anesthesiologists) put hundreds of thousands of people into chemical comas to enable other doctors to enter and alter our insides. Then they bring us back again. But quite how this daily extinction happens and un-happens remains uncertain. Researchers know that a general anaesthetic acts on the central nervous system – reacting with the slick membranes of the nerve cells in the brain to suspend responses such as sight, touch and awareness. But they still can’t agree on just what it is that happens in those areas of the brain, or which of the things that happen matter the most, or why they sometimes happen differently with different anaesthetics, or even on the manner – a sunset? an eclipse? – in which the human brain segues from conscious to not.

Nor, as it turns out, can anaesthetists accurately measure what it is they do.

For as long as doctors have been sending people under, they have been trying to fathom exactly how deep they have sent them. In the early days, this meant relying on signals from the body; later, on calculations based on the concentration in the blood of the various gases used. Recent years have seen the development of brain monitors that translate the brain’s electrical activity into a numeric scale – a de facto consciousness meter. For all that, doctors still have no way of knowing for sure how deeply an individual patient is anaesthetised – or even if that person is unconscious at all.

Anaesthetists have at their disposal a regularly changing array of mind-altering drugs – some inhalable, some injectable, some short-acting, some long, some narcotic, some hallucinogenic – which act in different and often uncertain ways on different parts of the brain. Some – such as ether, nitrous oxide (better known as laughing gas) and, more recently, ketamine – moonlight as party drugs. (“If you have an inclination to travel, take the ether – you go beyond the furthest star,” wrote the American philosopher-poet Henry David Thoreau after inhaling the drug for the fitting of his false teeth.) Different anaesthetists mix up different combinations. Each has a favourite recipe. There is no standard dose.

Today’s anaesthetic cocktails have three main elements: “hypnotics” designed to render you unconscious and keep you that way; analgesics to control pain; and, in many cases, a muscle relaxant (“neuromuscular blockade”) that prevents you from moving on the operating table. Hypnotics such as ether, nitrous oxide and their modern pharmaceutical equivalents are powerful drugs – and not very discriminating. In blotting out consciousness, they can suppress not only the senses, but also the cardiovascular system: heart rate, blood pressure – the body’s engine. When you take your old dog on its last journey, your vet will use an overdose of hypnotics to put him down. Every time you have a general anaesthetic, you take a trip towards death and back. The more hypnotics your doctor puts in, the longer you take to recover, and the more likely it is that something will go wrong. The less your doctor puts in, the more likely that you will wake. It is a balancing act, and anaesthetists are very good at it. But it doesn’t alter the fact that for as long as anaesthetists have been putting them to sleep, patients have been waking during surgery.

As Rachel’s caesarean proceeded, she became aware of voices, though not of what was being said. She realised that she was not breathing, and started trying to inhale. “I was just trying desperately to breathe, to breathe in. I realised that if I didn’t breathe soon, I was going to die,” she told me.

She didn’t know there was a machine breathing for her. “In the end I realised that I couldn’t breathe, and that I should just let happen what was going to happen, so I stopped fighting it.” By now, however, she was in panic. “I couldn’t cope with the pain. It seemed to be going on and on and on, and I didn’t know what it was.” Then she started hearing the voices again. And this time she could understand them. “I could hear them talking about things – about people, what they did on the weekend, and then I could hear them saying, ‘Oh look, here she is, here the baby is’, and things like that, and I realised then that I was conscious during the operation. I tried to start letting them know at that point. I tried moving, and I realised that I was totally and completely paralysed.”

The chances of this happening to you or me are remote – and, with advances in monitoring equipment, considerably more remote than 25 years ago. Figures vary (sometimes wildly, depending in part on how they are gathered) but big American and European studies using structured post-operative interviews have shown that one to two patients in 1,000 report waking under anaesthesia. More, it seems, in China. More again in Spain. Twenty to forty thousand people are estimated to remember waking each year in the US alone. Of these, only a small proportion are likely to feel pain, let alone the sort of agonies described above. But the impact can be devastating.

For Rachel, sleepless and terrified in her hospital room, it was the beginning of years of nightmares, panic attacks and psychiatric therapy. Soon after she gave birth, her blood pressure soared. “I was in a hell of a state,” she told me.

For weeks after she returned home, she would have panic attacks during which she felt she couldn’t breathe. Although she says the hospital acknowledged the mistake and the superintendent apologised to her, beyond that she does not recall getting any help from the institution – no explanation or counselling or offer of compensation. It did not occur to her to ask.

Things can go wrong. Equipment can fail – a faulty monitor, a leaking tube. Certain operations – caesareans, heart and trauma surgery – require relatively light anaesthetics, and there the risk is increased as much as tenfold. One study in the 1980s found that close to half of those interviewed after trauma surgery remembered parts of the operation, although these days, with better drugs and monitoring, the figure for high-risk surgery is generally estimated at closer to one in 100. Certain types of anaesthetics (those delivered into your bloodstream, rather than those you inhale) raise the risk if used alone. Certain types of people, too, are more likely to wake during surgery: women, fat people, redheads; drug abusers, particularly if they don’t mention their history. Children wake far more often than adults, but don’t seem to be as concerned about it (or perhaps are less likely to discuss it). Some people may simply have a genetic predisposition to awareness. Human error plays a part.

But even without all this, anaesthesia remains an inexact science. An amount that will put one robust young man out cold will leave another still chatting to surgeons. More than a decade ago, I found this quote in an introductory anaesthesia paper on a University of Sydney website: “There is no way that we can be sure that a given patient is asleep, particularly once they are paralysed and cannot move.”

Last time I searched, the paper had been adjusted slightly to acknowledge recent advances in brain monitoring, but the message remained the same: just because a person appears to be unconscious, it does not mean they are.

“In a way,” continued the original version of the paper, “the art of anaesthesia is a sophisticated form of guesswork. It really is art more than science … We try to give the right doses of the right drugs and hope the patient is unconscious.”

The death rate from general anaesthesia has dropped in the past 30 years, from about one in 20,000 to one or two in 200,000; and the incidence of awareness from one or two cases per 100 to one or two per 1,000. “Obviously we give anaesthetics and we’ve got very good control over it,” a senior anaesthetist told me, “but in real philosophical and physiological terms, we don’t know how anaesthesia works.”

It is perhaps the most brilliant and baffling gift of modern medicine: the disappearing act that enables doctors and dentists to carry out surgery and other procedures that would otherwise be impossibly, often fatally, painful.

The term anaesthesia was appropriated from the Greek by New England physician and poet Oliver Wendell Holmes in 1846, to describe the effect of the drug ether following its first successful public demonstration in surgery. Anaesthetise: to render insensible. These days there are other sorts of anaesthetics that can numb a tooth or a torso, simply (or unsimply) by switching off the nerves in the relevant part of the body. But the most widespread and intriguing application of this skill is what is now known as general anaesthesia.

The first public demonstration of the use of inhaled ether as a surgical anaesthetic in 1846 by an American dentist, William Thomas Green Morton. Photograph: Design Pics Inc/Rex/Shutterstock

In general anaesthesia, it is not the nerve endings that are switched off, but your brain – or, at least, parts of it. These, it seems, include the connections that somehow enable the operation of our sense of self, or (loosely) consciousness, as well as the parts of the brain responsible for processing messages from the nerves telling us we are in pain: the neurological equivalent of shooting the messenger. Which is, of course, a good thing.

I am one of the hundreds of millions of humans alive today who have undergone a general anaesthetic. It is an experience now so common as to be mundane. Anaesthesia has become a remarkably safe endeavour: less an event than a short and unremarkable hiatus. The fact that this hiatus has been possible for fewer than two of the 2,000 or so centuries of human history; the fact that only since then have we been able to routinely undergo such violent bodily assaults and survive; the fact that anaesthetics themselves are potent and sometimes unpredictable drugs – all this seems to have been largely forgotten. Anaesthesia has freed surgeons to saw like carpenters through the bony fortress of the ribs. It has made it possible for a doctor to hold in her hand a steadily beating heart. It is a powerful gift. But what exactly is it?

Part of the difficulty in talking about anaesthesia is that any discussion veers almost immediately on to the mystery of consciousness. And despite a renewed focus in recent decades, scientists cannot yet even agree on the terms of that debate, let alone settle it.

Is consciousness one state or many? Can it be wholly explained in terms of specific brain regions and processes, or is it something more? Is it even a mystery? Or just an unsolved puzzle? And in either case, can any single explanation account for a spectrum of experience that includes both sentience (what it feels like to experience sound, sensation, colour) and self-awareness (what it feels like to be me – the subjective certainty of my own existence)? Anaesthetists point out that you don’t have to know how an engine works to drive a car. But stray off the bitumen, and it is surprising how quickly pharmacology and neurology give way to philosophy: if a scalpel cuts into an unconscious body, can it still cause pain? And then ethics: if, under anaesthesia, you feel pain but forget it almost in the moment, does it matter?

Greg Deacon, a former head of the Australian Society of Anaesthetists, told me about a patient who was waiting to have open heart surgery. Deacon had been preparing to anaesthetise him, he said, when the man went into cardiac arrest. The team managed to restart the recalcitrant heart, then raced the patient into surgery, where they operated immediately. It was only once the operation had begun, the man’s heart now beating steadily, that they could safely administer an anaesthetic. It all went well, said Deacon, and the man made an excellent recovery. Some days later, the patient told doctors he remembered the early parts of the procedure before he was given the drugs.

“That is a sort of incidence of awareness which was thoroughly understandable and acceptable,” Deacon told me: he had not even known if the man’s brain was still working, let alone whether he would survive an anaesthetic. “We were trying to keep him alive.”

This is not denial. This is the tightrope that anaesthetists walk every day. They just tend not to talk about it.

In 2004, and against a backdrop of growing public and media concern, America’s Joint Commission on Accreditation of Healthcare Organizations finally issued an alert to more than 15,000 of the nation’s hospitals and healthcare providers. The commission, which evaluates healthcare providers, acknowledged that the experience of awareness in anaesthesia was under-recognised and under-treated, and called on all healthcare providers to start educating staff about the problem.

The American Society of Anesthesiologists subsequently acknowledged, in a 2006 practice advisory, that accidental intraoperative awareness, while rare, might be followed by “significant psychological sequelae … and affected patients may remain severely disabled for extended periods of time”.

Before that acknowledgment was published, however, then ASA president Roger Litwiller made a small but telling observation. Despite his organisation’s concern about anaesthetic awareness, he did not want the issue to be blown out of proportion: “I would also like to say that there is a potential for this subject of awareness to be sensationalised. We are concerned that patients become unduly frightened during what is already a very emotional time for them.”

This is the anaesthetist’s dilemma. Under stress – which affects just about everybody facing a general anaesthetic – we lose our ability and often desire to process complex information. More than half of all patients worry about pain, paralysis and distress. High anxiety or resistance to the idea of anaesthesia may even contribute to anaesthetics failing, or at least increase the chances that we will remember parts of the operation. The more anxious we are, the more anaesthetic it may take to put us to sleep.

This creates a quandary for doctors: how much to tell? When we are anxious, our bodies increase production of adrenaline-type substances called catecholamines. These can react badly with some anaesthetic agents. So what does an anaesthetist tell a patient who, because of the type of operation, or their state of health, is at higher than average risk?

“I mean, we’re trying to make people not worry about it,” said one Australian anaesthetist I spoke with, “but in the process I think we blur it so much that people hardly ever think about it, and that’s probably not right either … Should I be telling you that you’ve got a high risk of death? Is that going to frighten you to death?”

Today the profession makes much of the emergence of a new generation of anaesthetists who are more attuned to the experiences of their patients. But the reality is that anaesthetists remain for the large part the invisible men and women of surgery. Many patients still don’t meet them until just before – or sometimes after – the operation, and many, muffled in a fug of drugs, might not even remember these meetings. Nor do anaesthetists generally leave anything to show for their work: no scars or prognoses. When they do leave evidence, it is invariably unwelcome – nausea, a raw throat, sometimes a tooth chipped as the breathing tube is inserted, sometimes a memory of the surgery. It is unsurprising, then, that by the time an anaesthetist makes it into the popular media, he or she is generally accompanied by a lawyer.

For the doctors who each day make possible the miraculous vanishing act at the heart of modern surgery, this invisibility can be galling. It is not surgeons who have enabled the proliferation of surgical operations – numbering in the hundreds 170-odd years ago and the hundreds of millions today. It is anaesthetists. In hospital emergency rooms in Australia and other countries, it is not surgeons who decide which patient is most in need of and mostly likely to survive emergency surgery: anaesthetists increasingly oversee the pragmatic hierarchy of triage. And if you have an operation, although it is your surgeon who manages the moist, intricate mechanics of the matter, it is your anaesthetist who keeps you alive.

One of the first articles I came across when I started researching this subject was a 1998 paper by British psychologist Michael Wang entitled “Inadequate Anaesthesia as a Cause of Psychopathology”. Wang pointed out that pain – “even unexpectedly severe pain” – did not necessarily lead to trauma. Post-traumatic stress seldom followed childbirth, for example. What could be devastating, he said, was the totally unexpected experience of complete paralysis.

Even today, most patients undergoing major surgery have no idea that part of the anaesthetic mix will be a modern pharmaceutical version of curare, a poison derived from a South American plant, which causes paralysis. Few will be aware, either, that during surgery their eyes will be taped shut, that they may be tied down, and that they will have a plastic tube manoeuvred into their reluctant airway, past the soft palate and the vocal cords, overriding the gag reflex, and into the windpipe.

An anaesthetist checking a patient’s pupil to gauge the effect of an anaesthetic. Photograph: Cornell Capa/The Life Picture Collection/Getty Images

For the patient paralysed upon the table, said Wang, “[t]he realisation of consciousness of which theatre staff are evidently oblivious, along with increasingly frenetic yet futile attempts to signal with various body parts, leads rapidly to the conclusion that something has gone seriously wrong. The patient might believe that the surgeon has accidentally severed the spinal cord, or that some unusual drug reaction has occurred, rendering her totally paralysed, not just during the surgery, but for the rest of her life.”

As soon as anaesthetists explain to patients how the process works, it all starts to seem a lot less mysterious. And talk, it turns out, is not only cheap but effective: a preoperative visit from an anaesthetist has been shown to be better than a tranquilliser at keeping patients calm. I know from my own experience – I had surgery on my spine – how reassuring such a conversation can be. For me, it was not just the information; it was the fact of the human contact, of being treated as an equal, of being included, rather than feeling like an appendage to a process to which I was, after all, central.

Hank Bennett, an American psychologist, remembers a young girl whose mother brought her to see him some time after the girl had her adenoids removed. The surgeon referred the mother to Bennett after she had returned to him in a state of anxiety about her child. The surgery had been straightforward, but the mother felt that something was very wrong with her previously happy daughter: the child had withdrawn from her family and friends, and had stopped working at school. She could no longer fall asleep without her mother sitting with her, and was afraid of the dark.

Bennett spoke with the girl. He told her there must be a reason she had changed her behaviour, and asked if it might have something to do with the operation.

Bennett recalled: “And she said, ‘Yes. They said that they were going to put me to sleep, but the next thing I knew, I couldn’t breathe.’ Now, she was only momentarily like that – she does not remember the breathing tube going in – but when I asked why she was doing these things differently at school and at home, she said: ‘Well, I have to concentrate and I can’t be bothered by anything. I’ve got to make sure that I can breathe.’”

Bennett referred the girl to a child psychologist, and within weeks she was back to herself. Today she would be approaching middle age. “But let’s say that was just luck,” Bennett says now. “What if nothing had been picked up about that? Would she have been permanently changed? I think that you would say, yes, she probably would have been.”

So if you were my anaesthetist and I your patient, there are some other things I’d hope you would do in the operating theatre. Things that many already do. Be kind. Talk to me. Just a bit of information and reassurance. Use my name. Patients who remember waking are often greatly relieved at having been told what was happening to them, and reassured that this was OK and that they would now drift back to sleep.

The Fifth National Audit Project on accidental awareness during general anaesthesia states: “The patient’s interpretation of what is happening at the time of the awareness seems central to its later impact; explanation and reassurance during suspected accidental awareness during general anaesthesia or at the time of report seems beneficial.” Hospital staff could put a sign on the wall of the operating theatre: “The patient can hear”. Because one of the strange things about anaesthetic drugs is that they can exert their effect in each direction – not just upon the patient, but upon the doctors and theatre staff performing the procedure.

After the teenage son of a good friend was badly burned in an accident some years ago, he had to endure weeks of intense pain, culminating each week in the agonising ritual of nurses changing the dressings on his chest and arms. They did this by giving him a dose of a sedative drug designed to distract him from the pain and prevent him remembering it. My friend would attempt to comfort her son as he yelled and as the nurses got on with their difficult task. What she observed was that while the drugs did give her son some distance from his pain, and certainly his memories of it, they also gave the nurses some distance from her son. It was an understandable, perhaps necessary, distance; but inherent in that tiny retreat (the lack of eye contact, the too-bright voices) was a loosening of the tiny filaments that connect us one to another, and through which we know we are connected.

It is a process inevitably magnified in the operating theatre, where the patient is silent and still, to all intents absent, and where their descent into unconsciousness is routinely accompanied by the sounds of the music being cranked up (one prominent Australian surgeon is said to favour heavy metal), and conversation. It need not take a scientific study to tell us that this deepening of respect and focus is good not only for patients, but for doctors, too. In the end, it might not even much matter what you say. During an operation, “a soothing voice may be more important than what the voice says,” writes psychologist John Kihlstrom, who still encourages anaesthetists to talk to their anaesthetised patients (“about what is going on, giving reassurance, things like that”) but acknowledges that he doesn’t expect them to understand any of it – not verbally at least.

Japanese anaesthetist Jiro Kurata calls this “care of the soul”. In an unusual and rather lovely paper delivered at the Ninth International Symposium on Memory and Awareness in Anaesthesia in 2015, he wondered if there might be “part of our existence that cannot ever be shut down, which we cannot even conceive by ourselves” – a “subconscious self” that might be resistant to even high doses of anaesthetics. He called this the hard problem of anaesthesia awareness. I have no idea what his colleagues made of it. But his conclusion seems unassailable.

“Any solution? Science? Yes and no. Monitoring? Yes and no. Respect? Yes. We must not only be aware of the inherent limitation of science and technology but, most importantly, also of the inherent dignity of each personal ‘self’.”

Anaesthesia: The Gift of Oblivion and the Mystery of Consciousness by Kate Cole-Adams (Text Publishing Company, £12.99) is published on 22 February. To order a copy for £9.99, go to guardianbookshop.com

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.