Next year, if all goes to plan, a dozen patients with clinical depression will be invited to a UK laboratory and given psilocybin – the psychedelic ingredient found in magic mushrooms. Over the next four or five hours, many of these volunteers will experience dream-like euphoria as colours, smells and sounds become more intense, perception of time distorts and their sense of self dissolves. Some may feel a surge of electricity through their bodies, sudden clarity of thought or hilarity. Others may experience anxiety, confusion or paranoia. These hallucinogenic effects will be short-lived, but the impact of the drug on the volunteers could be long-lasting.

There is tentative evidence that psilocybin, along with other psychedelic drugs, can "reset" abnormal functioning of the brain if given in a safe, controlled way as part of therapy. For those raised on the post-1960s dogma that magic mushrooms and LSD unleash mental illness, trigger flashbacks and cause personality changes, the idea that they could actually cure disorders of the brain is mind-blowing.

The pilot study will involve patients who have failed to respond to conventional treatment and is the idea of Professor David Nutt and Dr Robin Carhart-Harris at Imperial College's Neuropsychopharmacology Centre in London. They argue that psychedelic drugs could prove beneficial to millions of people and that it is time to end the 50-year stigma surrounding their therapeutic use. Nutt and Carhart-Harris have already used MRI scanners to study changes in the brain while 15 volunteers took psilocybin. A similar study on 20 volunteers given LSD has just finished.

"As a non-clinician, I was convinced by seeing how psilocybin affects the brain," says Carhart-Harris. "It was quite stark how similar it was to the existing treatments for depression."

The Imperial scientists are at the forefront of a hallucinogenic research revival and say they have 50 years of science to catch up on. In the 50s and 60s you could barely open a psychiatric journal without coming across a paper on LSD. Then, LSD was the latest wonder substance, with potential to treat depression, addiction and headaches. Between the late 1940s – when it was made available to researchers under the name Delysid – and the mid-60s there were 1,000 academic papers investigating its effects on 40,000 people.

But as the drug grew in popularity among recreational users – and became linked to the counter-cultural revolution and Vietnam protest – the backlash began. By the late 60s it was at the heart of a full-blown moral panic and the US government cracked down on it.

Nutt, who was controversially sacked as chair of the UK government's Advisory Council on the Misuse of Drugs in 2009 for claiming that horse-riding was more dangerous than ecstasy, says the justification for banning LSD and hallucinogens was a "concoction of lies" about their health impacts, combined with a denial of their potential as research tools and treatments.

"It was unquestionably one of the most effective pieces of disinformation in the history of mankind," says Nutt. "It led to a lot of people believing these drugs were more harmful than they were. They are not trivial drugs, but in comparison with drugs that kill thousands of people a year, like alcohol, tobacco and heroin, they have a very safe track record and, as far as we know, no one has died."

In 1971 the UN convention on psychotropic substances classified LSD, other hallucinogens and cannabis as schedule 1 drugs: dangerous substances with no medical benefit. In contrast heroin – an addictive and far more dangerous drug – was classed as a less restricted schedule 2 drug because it had known pain-killing effects.

In order to study a schedule 1 drug, UK academics need a £3,000 Home Office licence and years of form-filling. They must also source legal but not overtly expensive supplies of the drug. The red tape – and the need to persuade sceptical university ethic committees – proved stifling for research. "It's a catch-22," says Carhart-Harris. "It's difficult to study LSD and psilocybin to see if they have medical use because they are schedule 1. And they are only classed as schedule 1 because they are deemed to have no medical use."

But things are changing. The first two papers on LSD experiments since the 1970s were published this year: one by the Imperial team; the other by Swiss researchers on easing anxiety among terminally ill cancer patients. In the UK the revival of academic research into "recreational" drugs is being driven by the Imperial scientists working with the Beckley Foundation, a charity run by English aristocrat Amanda Feilding, the Countess of Wemyss and March. After taking LSD in the 1960s she became fascinated with its potential for creativity and enhancing understanding.

Her foundation supports and initiates research into psychoactive substances – including LSD, magic mushrooms and cannabis, a plant used in medicine for thousands of years. "By prohibiting research into this category of substance, because of a social misconception, we are depriving suffering ill people from a potential treatment which has a very long history," she says.

The Imperial scientists stress they are studying psychedelic drugs in safe environments to reduce the risk of negative experiences. The drugs used are chemically pure and given in controlled doses.

"Self-medication is definitely a no-no from my perspective," says Carhart-Harris. "These drugs are powerful and the therapeutic model we are going to adhere to is quite specific in that it emphasises that the drug needs to be taken in the right environment and with the right support. We have professional psychotherapists there who are trained and understand all the eventualities of what might happen, and so I think it would be reckless for people to try to do it by themselves."

LSD (lysergic acid diethylamide)

Swiss chemist Dr Albert Hoffman created LSD in 1938 from a substance in ergot fungus. After its hallucinogenic properties were discovered by Hoffman in 1943, it was used as a tool for modelling schizophrenia in healthy volunteers and taken by some psychiatrists to gain insight into mental illness. By the early 60s it was being given to patients undergoing psychotherapy.

Many of the 1,000 or so academic papers from this era discuss its use in treating depression. Dr Robin Carhart-Harris says the evidence from historical research was weak.

"Most of this stuff was anecdotal," he says. "They wouldn't have had a control group and the outcomes would be poorly defined." There is much stronger evidence that LSD can treat addiction. A 2012 meta-analysis of six controlled trials from the 50s and 60s found its clinical efficiency for the treatment of alcohol addiction to be as effective as any treatment developed since. It has also been shown to help patients who are terminally ill come to terms with dying. The mechanisms of LSD are still poorly understood. It seems to mimic some actions of the brain chemical serotonin, which is involved in memory formation, mood and reward, but how it triggers such powerful altering effects isn't clear.

The Imperial/Beckley MRI research showed that brains of volunteers on LSD become less organised and more chaotic, while parts of the brain that would not normally communicate with each other link up. In this disorganised dream-like state, the brain is open to new leaps of creativity and flights of fancy. Dr Carhart-Harris believes that hallucinogens may temporally "loosen" the rigid structures of the brain, which have developed as we age. An acid trip is a bit like shaking up a snow globe. This loosening could help the brain break the cycles of addiction and depression.

There are risks with LSD, of course, but Professor David Nutt says they are often exaggerated. The effects of taking LSD are unpredictable: users can lose their sense of judgment - and put themselves in risky situations. A minority may experience flashbacks some time after tripping; in others the experience may set off mental health problems that have previously gone unnoticed.

Cannabis

Cannabis has been taken medicinally for thousands of years. It was used in the 19th century to treat pain, spasms, asthma, sleep disorder, depression and loss of appetite, and was even recommended by Queen Victoria's doctor. Two chemicals in cannabis attract interest from scientists - tetrahydrocannabinol (THC), the main psychoactive compound, which has some pain-relieving properties, and cannabidiol (CBD), which appears to confer health benefits without the high.

More than 100 trials of cannabis or cannabis-derived substances have taken place since the 1970s. A cannabis extract mouth spray marketed as Sativex was approved for use in multiple sclerosis in 2011 after a trial found it improved sleep, reduced spasticity and the number of spasms. In the 1970s and 80s, trials showed drugs derived from cannabis could help with nausea in cancer patients undergoing chemotherapy. In the 1990s, research showed it could prevent anorexia in HIV patients. There is also evidence that cannabis can relieve chronic pain when taken with other medications.

In the US, 20 states allow the possession and use of cannabis for medical reasons, while two states – Colorado and Washington – have decriminalised it. It has been legal for medical use in the Netherlands for more than 25 years.

Some researchers believe cannabis has potential as a treatment for attention deficit hyperactivity disorder, post-traumatic stress and insomnia.

One of the difficulties with cannabis research is that the nature of the street drug has changed over the last few decades – partly in response to demand from users for a bigger high. Studies at the Institute of Psychiatry at King's College London have shown that THC increases anxiety and short-term psychotic symptoms, while CBD - which appears to have most of the medical benefits - has the opposite effect. Modern street varieties of cannabis, or skunk, are high in THC and extremely low in CBD.

Magic Mushrooms

Like LSD, magic mushrooms are hallucinogenic. The active compound is psilocybin and, like LSD, it may be useful in treating depression. A study of 15 volunteers by Dr Robin Carhart-Harris of Imperial College London showed that the compound helps suppress the part of the brain often hyperactive in depression called the medial prefrontal cortex. This region is linked to introspection and obsessive thinking.

It is too soon to say whether magic mushrooms can treat depression, though. In 2013, Imperial College scientists were awarded a Medical Research Council grant to study the effects of psilocybin on a dozen patients with depression. The study is being hampered by red tape, but should start next year.

Psilocybin may also be useful in treating addiction. A Beckley Foundation-funded study at Johns Hopkins University gave 15 people trying to give up smoking two to three rounds of the drug. Six months later, 12 remained off cigarettes – a success rate of 80%. The best smoking cessation drugs are 35% effective. While promising, a much bigger study is needed.

The best smoking cessation drugs are only 35% effective. Some users of magic mushrooms say it helps with obsessive compulsive disorder. So far there has been just one clinical trial,involving nine people. The results were again promising – but larger studies are needed.

There is also some evidence that it can help cancer patients come to terms with their condition.

According to Professor David Nutt, the risks associated with magic mushroom use are relatively low compared to drugs such as alcohol, tobacco or heroin. On a scale of risk published in the Lancet in 2010, magic mushrooms came bottom out of 20 of the most commonly used drugs.

MDMA (Ecstasy)

First synthesised 100 years ago, MDMA was used in the 1970s during psychotherapy but exploded into the public consciousness with the arrival of the dance culture in the late 1980s.

It works by increasing the activity of three neurotransmitters, or chemical messengers, in the brain: serotonin, dopamine and noradrenaline. The high levels of serotonin generate a feeling of euphoria, affection and goodwill.

A small controlled studying in the US showed that 80% of people with post-traumatic stress disorder benefited from MDMA during therapy. The current treatment for PTSD involves patients reliving their stressful experiences. MDMA seems to work by artificially raising patients' moods, and making it easier for them to recall and discuss the experiences that caused PTSD.

Professor David Nutt says MDMA may also have benefits helping with end-of-life anxiety, and couples therapy. It could also help with Parkinson's disease. More research is needed on the risks. however. Animal studies suggest it may cause long-term damage to the brains of rats, but the evidence in people is inconclusive. Between 1997 and 2012, 577 deaths were linked to ecstasy in England and Wales – mostly from heatstroke, heart problems or excess water consumption.