A symphonic note lit three strands of deep-red light trickling like water in my right visual field. Deeper tones were huge blue clouds in the middle distance, pouring from above. A prolonged violin solo turned the sky yellow, and brought with it a comet's tail of body parts flying from the upper left of my visual field to the lower right, disappearing behind me.

This happened within the first hour of my swallowing a capsule of psilocybin, the psychoactive ingredient in "magic mushrooms". I'd volunteered to be a scientific-research-study subject at the Johns Hopkins University School of Medicine's Behavioural Pharmacology Research Unit (BPRU) as part of a clinical trial to test the hypothesis that psilocybin can help cancer patients to regain a sense of existential meaning in the face of their disease. This study is designed to measure how well the drug does in alleviating the anxiety and depression associated with cancer. There are currently only two such clinical studies under way using this drug, both of them in the US. Another is scheduled to begin here in the UK early next year. The drug is not available legally for any other use in the UK or in the US.

I was diagnosed with breast cancer in 2009. A year on from a lumpectomy and radiation therapy, now on aromatase inhibitors, my prognosis seemed good. But rather than cheerfully getting on with life, I was spending most days at my desk crying. I began searching the web for a way to kill myself that wouldn't be either messy or too painful.

Both here in the UK and in the US, where I had my surgery, medicine excels at finding cures for disease and saving lives. All that excellence has created a kind of void, wherein treatment of the disease has trumped the human needs of those being treated. My demoralisation is common among cancer patients. We obsess about survival and what the future holds for us. The system isn't structured to help doctors to help us.

As a result, it's a good thing for me that after a long banishment from the clinical scene, hallucinogens are making a comeback. Prior to their 1970s prohibition, psilocybin and LSD were prescribed to around 40,000 research subjects, among them people with cancer, alcoholics, and those suffering from depression and obsessive-compulsive disorder. The results showed remarkable promise in helping people to overcome pain, fear, compulsive behaviours and psychological isolation. Following a dose administered in a proper clinical setting, hallucinogens diminish symptoms and improve therapeutic outcomes for months, even years, afterwards. Many patients undergo a psychospiritual epiphany, wherein they feel a union with the Universe and sense life has meaning beyond what's happening in the physical body.

Scientists have been waiting a long time for the tide to turn. "We had 30 years to practise exactly what we tell study subjects before their psilocybin sessions," said Bill Richards, one of the researchers at the BPRU who successfully treated terminally ill patients with hallucinogens at the Maryland Psychiatric Research Centre from the early 1960s until the centre was shut down in 1977. "Trust, let go, be open."

Volunteers for the psilocybin scientific- research trial visit the Baltimore clinic twice, receiving a low dose of psilocybin on one visit and a moderately high dose on the other. We're screened for mental, emotional and physical problems before we're officially admitted into the study. Giving psilocybin to people who have underlying psychotic disorders or schizophrenia can be catastrophic. Many volunteers are sent home after a day.

"Set and setting" are key factors in the hallucinogen-assisted therapy equation. "Set" refers to "mindset" – the patient's mental and emotional attitude towards the hallucinogenic experience. "Setting" is the physical and social environment – the room or space itself, and the people who are present with the subject during the experience.

"Set" must include a willingness to move towards repellent or frightening thoughts and images, rather than trying to flee from them. Part of the "set-and-setting" equation includes complete trust in the guides, trained psychotherapists who remain throughout the entire session. The images and feelings can be beautiful and transcendent, or terrifying and disgusting – or all of these over the course of the day.

Four different psychotherapists, and the study's co-ordinator, interviewed me for hours. They asked questions such as: "If you had to spend the day being nauseated, could you tolerate it?" and "Do you ever think there are people or other beings who are transmitting secret coded messages to you alone?"

I filled in more than 50 pages of questionnaires, including assessments of optimism and pessimism, pain scales, depression scales, queries about my lifestyle and my habits, as well as one called Assessment of Spirituality and Religious Sentiments. My blood pressure was monitored every 15 minutes for two hours. The physical exams often uncover maladies subjects themselves aren't aware of. Borderline diabetes, heart arrhythmia or slightly off-par liver function will get you excluded, as well as traces of alcohol or drugs. All feelings and experiences – from childhood trauma to attitude to internal conflicts – are subjected to scrutiny. The guides have to know as much as possible so they can be supportive if complex or painful feelings arise. They often do.

I met with Roland Griffiths, professor of behavioural biology and of neuroscience and the study director at the BPRU, after each day's battery of tests.

"You have to approach this experience with radical curiosity," he said. Then he'd ask me: "What do you think about God? Where do we go when we die?" I didn't know the answers.

By the end of the third day, I felt as though I had been flayed. But I'd passed, and I was in.

When my session day arrived, I was brought into a softly lit, comfortably decorated lounge, invited to lie down on a sofa and listen to music. Then I swallowed a purple capsule of psilocybin.

Some time later, when I was deeply within the world of the drug and the imagery it evoked, I found myself inside a steel industrial space. I became aware of my animosity towards my two living siblings. A woman sitting at the end of a long table, wearing a net cap, white clothes, and working busily, turned and handed me a Dixie cup.

"You can put that in here," she said. So I did. The cup filled itself with my bilious, sibling- directed feelings. "We'll put it over here," she said, and placed it on a table at the back of the room. Then she went matter-of-factly back to work, along with now-numerous busy women. At that point, my guide Fred asked me what was happening. I recounted the scene and then I began to laugh out loud. My own laughter appeared to me in a midnight-blue, cloud-dark sky as an effusion of twinkling gemstones. I was in two places at once, both in the session room, talking to my guide, and in the other world of the drug, with its own aesthetics and its own logic.

Not all subjects have an all-encompassing transcendent experience during their sessions, wherein they feel a profound oneness of all things, a union with the Universe or with God. I did not – and was at first disappointed. But as the months have passed, I realise what I did gain is immeasurable. Since my session, my mood has improved, and my sense of myself, as a person occupying a certain space in the Universe, has altered. Later on, when talking about my hallucinations with the clinicians and my guides, I found they provided me with some profound truths about my life, my feelings and my sense of myself. My tendency to judge myself with a kind of murderous harshness has ebbed. I'm now able to feel more compassion towards myself. I no longer spend days worrying about the future, and about whether I'll have a cancer recurrence, or whether I'll die alone.

Psilocybin works by providing a neurochemical bridge between spiritual guidance and talking therapy. The drug's therapeutic value depends entirely upon the patient's feelings and perceptions during the session and the way he or she processes the memories afterwards. The drug allows patients a mini-holiday from their own egos, a span of time to exist without that nattering part of us that constantly worries about things such as: "How will I survive now that I don't have a job?" and "I wonder if my ex has started seeing somebody else."

"The drug is a skeleton key which unlocks an interior door to places we don't generally have access to," Dr Richards said. "It's a therapeutic accelerant."

Treatment with hallucinogens cannot – and probably will not ever – be given as a daily pill. Patients will have one or possibly two sessions in a clinic or hospital setting, and no more. The powerful images and emotions require preparation and guidance throughout the experience, which generally lasts from four to six hours. Set and setting have to be honoured.

Giving hallucinogens any other way is "a recipe for paranoia, anxiety, and disaster", Dr Richards said. The therapeutic value of these drugs depends entirely upon the patient's feelings and perceptions during the session and the way he or she processes the memories afterwards. Patients who undergo a transcendent peak while taking psilocybin describe it as among the most meaningful events in their lives.

As the population ages, and more people face end-of-life issues and prolonged illness, the demand for improved palliative care and the inclusion of psychospiritual considerations is altering the way medicine will be practised.

In the UK, a clinical trial using psilocybin to treat depression is due to begin in January 2013. David Nutt, director of the Neuropsychopharma-cology Unit in the Division of Brain Sciences at Imperial College London, is in charge. When I asked him about the therapeutic obstacles presented by legal bans against psychoactive drugs, he said: "We'll change the law."