On a hot evening in June, in a crowded room above a London pub, Dr Robin Carhart-Harris, a research associate in the Centre for Neuropsychopharma-cology at Imperial College, is giving a public talk about his work. He is having to make himself heard over the boozy commotion downstairs, where people are watching Chile put Spain out of the World Cup. But there is a slightly giddy atmosphere in the function room, too, because the doctor's area of research is as exciting as it is taboo: he is investigating the brain effects and potential therapeutic uses of psychedelic drugs.

Carhart-Harris is the first person in the UK to have legally administered doses of lysergic acid diethylamide (LSD) to human volunteers since the Misuse of Drugs Act of 1971, and his presentation climaxes with a slide showing something no one else has seen before: an as-yet unpublished cross-sectional image of the brain of a volunteer who was in an fMRI scanner while tripping on acid. Blobs of colour indicate changes in blood flow, from which can be inferred changes in levels of activity in specific brain regions – notably, in this case, the hippo-campus, which is involved, among other things, in making memories and giving them context.

"We've only looked in six brains so far," says Carhart-Harris when we meet a couple of weeks later in a coffee shop near his flat in Notting Hill. Born in Durham 33 years ago and raised in Bournemouth, he has an indie haircut and looks a bit like he could be the celebrity physicist Brian Cox's cooler younger brother. He is a careful and articulate speaker, but his enthusiasm for his work is evident. "We're at an early, but certainly promising, stage. It's really exciting," he says.

The potential scientific benefits of psychedelics (as distinct from whatever cultural, social, artistic, spiritual or subjectively enjoyable benefits one might also argue they have) fall broadly into two categories. They look like being medicinally or therapeutically useful, and they offer an unconventional view of the workings of the human mind, such that the age-old, so-called "hard problem of consciousness" might be made a little easier. The etymology of the word "psychedelic" is, after all, from the Greek for "mind-revealing".

Plant-derived psychedelics such as mescaline (from the peyote cactus), DMT (from the root of the ayahuasca vine) and psilocybin (magic mushrooms) have been used therapeutically and medicinally for millennia. In 1943, however, Albert Hofmann, a 37-year-old Swiss chemist in the laboratories of the pharmaceutical company Sandoz in Basel, accidentally ingested – through his fingertips – a chemical he had synthesised from the ergot fungus, and became the first person to experience its remarkable mind-altering properties. Interviewed shortly before his 100th birthday, he called LSD "medicine for the soul".

A trip through time: The history of LSD Show all 5 1 /5 A trip through time: The history of LSD A trip through time: The history of LSD 19 April 1943 Having accidentally ingested LSD three days earlier, Albert Hofmann takes the world's first intentional acid trip and rides home from the lab on his bike. The event is commemorated annually on "Bicycle Day" AFP/Getty Images A trip through time: The history of LSD May 1950 The first article about LSD appears in the American Journal of Psychiatry A trip through time: The history of LSD 22 November 1963 Aldous Huxley, author of The Doors of Perception, instructs his wife to administer him with LSD on his deathbed, and passes away "very, very gently" AP A trip through time: The history of LSD April 1965 The Beatles are introduced to acid by George's dentist. Sgt. Pepper's Lonely Hearts Club Band comes out in June 1967, and while John denies "Lucy in the Sky with Diamonds" was an intentional expansion of "LSD", few believe him Rex Features A trip through time: The history of LSD 14 January 1967 Four years after he is sacked from the psychology department at Harvard, acid evangelist Dr Timothy Leary tells a 30,000-strong gathering at the Human Be-In in San Francisco to, "Turn on, tune in, drop out" Getty Images

Uniquely potent in minute doses, and with what Carhart-Harris calls "a very favourable physiological safety profile" – which is to say, it is non-toxic – this newly synthesised psychedelic drug opened new doors, in more ways than one. "You could say the birth of the science of psychedelics occurred with the discovery of LSD," says Carhart-Harris. "It was only then that we started to study them systematically."

By 1947, Sandoz was promoting LSD, under the brand name Delysid, as an adjunct to psychotherapy. Cary Grant famously used it during his therapy, as did the Alcoholics Anonymous co-founder Bill Wilson. Between the 1950s and 1965, when Sandoz withdrew the drug, there were more than 1,000 clinical papers discussing 40,000 patients. A 2012 meta-analysis of six controlled trials from the era found its clinical efficiency for the treatment of alcohol addiction to be as effective as any treatment developed since.

"I personally think it has a great deal of potential for treating addiction," says Carhart-Harris. "It's slightly hypothetical, but it's based on what we know about the way the brain works, which is that it settles into configurations of activity that seem to underly certain psychopathologies.

"Depression and addictions rest on reinforced patterns of brain activity, and a psychedelic will introduce a relative chaos. Patterns that have become reinforced disintegrate under the drug. I've used the metaphor of shaking a snowglobe. And there's some evidence that psychedelics induce plasticity, in terms of neural connections in the brain, such that there is a window of opportunity in which connections can either be broken or reinforced. New things can be learnt at the same time that old things can be unlearnt. It induces a kind of suppleness of mind."

The biological mechanisms that underlie the psychedelic experience were unknowable in the 1950s and 1960s, prior to the modern era of neuroscience and brain imaging. "Rather than testing whether a drug works, finding that it seems to, and then trying to understand how, I prefer the way we're doing it now," says Carhart-Harris. "It's a bit more logical and cautious."

Caution, readers might be pleased to note, is a watchword throughout our conversation. "The dangers with psychedelics – and there are potential dangers," says the doctor, " arise when they are taken without the proper caution. The model for how the drugs are taken therapeutically is very different from how people might take them recreationally. People are in a particularly sensitive and vulnerable state on psychedelics, and I do think you need that professionalism and structure to have it done properly."

Albert Hofmann's 1980 book traces LSD’s path from a promising research medicine to a recreational drug that sparked hysteria (Kalpesh Lathigra)

The doses Carhart-Harris administers are lower than a recreational user might typically take, but if anything the volunteers' experiences would seem to be more vivid. "When people take psychedelics recreationally, in a social context," he says, "they might get preoccupied by the perceptual changes and the novelty, and they'll laugh their way through with a certain amount of confusion and anxiety. But in an experimental context, particularly in the therapeutic context, people lie on a couch with their eyes closed and have a very introspective experience. It's richer; psychologically, it's more interesting. Without distractions, emotions and memories are more likely to emerge spontaneously. There is a possibility of having quite vivid recollections of past experiences, such that they're not past any more and can be re-experienced in the present. It gets very interesting when people start describing where they're going in their minds. It's the kind of stuff you just wouldn't hear ordinarily."

Music is used in both the therapeutic and the brain-imaging studies. "People typically play classical music in psychedelic-assisted psychotherapy," says Carhart-Harris, "But we're going with some relaxing ambient stuff, particularly in the setting of a noisy MRI scanner. We're also looking at the interaction between music and LSD; seeing whether emotional arousal and ego dissolution are enhanced. The theory – and it k is only a theory because it's never been tested until now – is that the music can do a number of things. It can have a steadying influence, but it can also help facilitate emotional release."

The subjects, who have mostly been men – "We need more female volunteers," says Carhart-Harris – have largely reported positive experiences. But, he adds, "We've had some tears, some people who've been anxious and at least one who hasn't enjoyed it." One subject recalled a relationship break-up, which was "melancholic but also moving, and not necessarily negative".

Last month, the Daily Mail reported on Carhart-Harris's research into psilocybin and its potential for treating depression. I suggest that the comments below the article ("Duh, but your brain will be fried and later on you'll be stupid" or "How can you be depressed if you're dead or in a mental institution?") might offer a representative sample of commonly held fears. But Carhart-Harris isn't sure. "That's an example of primitive thinking, really, where instead of being rational and assessing the evidence, one is biased by one's emotions – in this case, one's fear about the unknown."

Having said that, "There have been some cases in the past of psychosis being induced by psychedelics. More so when the drugs are taken recreationally. It hasn't happened in modern trials, but it was reported in a few cases in the 1960s. We recruit only people who have previous experience with psychedelics. If our main aim is to find out how the drugs work in the brain, we may as well play on the safe side and get volunteers who've previously tolerated them."

LSD is a Schedule I drug under the 1971 UN Convention of Psychotropic Drugs, and a Schedule I, Class A drug under the UK Misuse of Drugs Act. But this categorisation, which both earns the drug its forbidding reputation and hinders research into it, is something of a historical accident rather than an accurate reflection of the dangers it poses.

Robin Carhart-Harris in his office, where he is investigating the loss of sense of self induced by psychedelics (Kalpesh Lathigra)

Heroin, for example, is a lesser, Schedule II drug under the 2001 Amendment to the UK Act, because while it is more dangerous than LSD and has a high potential for abuse, its usefulness as a pain medication had long been recognised. To stock LSD, which was deemed not to have any medical uses, on the other hand, requires a licence from the Home Office. Such licences have been prohibitively expensive and time-consuming for researchers to obtain; only four have ever been granted. You also have to find a lab willing and able to manufacture the drug, and there are similarly few of them, for the same reasons.

Professor David Nutt, the director of the Neuropsychopharmacology Unit where Carhart-Harris is doing the research ("The buck stops with me, but Robin's doing all the legwork and I'm just taking the credit," he says), has written extensively about the wrong-headedness of the current drug laws. "It is absurd to treat LSD and psilocybin as more dangerous than heroin," he tells me. In a 2010 study published in The Lancet, following on from the work that led to his controversial sacking from the chair of the Government's Advisory Council on the Misuse of Drugs the year before, Professor Nutt ranked the 20 most commonly used drugs according to the evidence for the overall amount of harm they do to the users and to society. Psilocybin came last, with an overall harm score of six, and LSD came 18th with a score of seven, compared with alcohol's 72 and heroin's 55.

When I ask why LSD was prohibited, he has a surprisingly simple three-word answer: "the Vietnam war". Essentially, when its use spread to the general population in the mid 1960s, "Young Americans realised they didn't want to fight any more. That brought a huge tension into society. So they had to create reasons for banning the drug. Everyone knew the arguments were totally specious. But no one stood up."

Carhart-Harris has a further explanation: "Psychedelics are scary because they reveal the mind, and people are scared of their own minds. They're scared of the human condition, really."

The brain and mind both have a hierarchical structure, and what Freud intuited and neuroscience has confirmed is that a significant portion of our mental activity occurs at levels other than the one at which your normal, waking, conscious self – your ego – is operating. As a result and by their very nature, these unconscious processes have traditionally been hard to investigate. But the non-normal consciousness that psychedelics induce is characterised by a dissolution of the ego; a loss of the sense of self. "That is why they are so valuable as tools to understand the mind," says Carhart-Harris. "This division between the ego-intact mode of consciousness and a mode which is more primitive breaks down, so you can begin to observe the unconscious mind."

No wonder the Czech psychiatrist Stanislav Grof, in 1975, predicted that "psychedelics, used responsibly and with proper caution, would be for psychiatry what the microscope is for biology or the telescope is for astronomy". Unfortunately, the effective prohibition of LSD research meant it was not to be. Perhaps Professor Nutt has Grof in mind when he says: "I think it's the worst censorship of research since the Catholic Church banned the telescope." He continues: "There's a lot of evidence for LSD being an efficacious treatment for things such as addiction. But no one's done a study on it for 50 years. This is outrageous."

Dr Carhart-Harris says: 'Criminalising these substances doesn't help anyone. Historically they've benefited mankind' (Kalpesh Lathigra)

Perhaps because he sees such potential in them, or perhaps because he suffered the latest in a line of funding knock-backs in the week that we meet, Carhart-Harris sounds somewhat peeved that psychedelics remain an outsider scientific interest. "I've been going for 10 years and done a lot of, I think it's fair to say, quite pioneering research. But it does feel as though there's some kind of conservative resistance k to it. People may claim to be supportive of novel research, but are they really?" he wonders.

Psychedelics certainly have their devotees. One such is Amanda Feilding, Countess of Wemyss and March, who in 1998 founded the Beckley Foundation, a charitable trust with the twin aims of initiating and conducting research into psychedelics, and effecting changes to global drugs policy. The foundation has provided a major part of the funding for the LSD study. "She has been critical to our progress," says Carhart-Harris, "and consistent in her support."

"I've been wanting to do this research for 20 years," Feilding tells me – and adds that she had personally promised Albert Hofmann she would get an LSD study off the ground in the modern era and in his lifetime. ("He was truly delightful. Very small, but a remarkable giant of a man." Unfortunately he died in 2008, aged 102.)

"I thought it essential to break the taboo on these substances," Feilding explains. "And hopefully, slowly, by doing good work with the very best experts on both policy and science, we can build up an evidence base which will inform people that these substances have potential benefits, and that criminalising them doesn't help anyone. Because historically they had always benefited mankind. They used to be sacraments, they were termed the flesh of the gods but now they're [considered] the substance of the devil."

Feilding "has been quite a lone voice, really", says Carhart-Harris, which means he also has to consider other sources of research funding. "I'm now thinking about crowd-funding," he says. "There's so much enthusiasm out there: I get inundated with emails from people telling me that they believe in its potential."

That potential is becoming clear, yet the barrier to progress is not scientific but legal, and depends on political will. "It's very easy to get caught up in the optimism of it all," says Carhart-Harris, "but the more psychedelics become visible, the more likely it is that there's going to be some kind of conservative reaction to them.

"Psychedelics are polarising, and people are opposed to them because they're threatening. But science is an exercise in honesty, really. That's its great merit. And if you want to understand reality better, then you have to confront things that might be difficult."

For more information and to donate towards the research: http://psychedelicscience.org.uk

A trip through time: The history of LSD

19 April 1943

Having accidentally ingested LSD three days earlier, Albert Hofmann takes the world's first intentional acid trip and rides home from the lab on his bike. The event is commemorated annually on "Bicycle Day"

May 1950

The first article about LSD appears in the American Journal of Psychiatry

13 April 1953

The CIA initiates the MKULtra Project to investigate LSD's potential as a mind-control drug

22 November 1963

Aldous Huxley, author of The Doors of Perception, instructs his wife to administer him with LSD on his deathbed, and passes away "very, very gently"

Summer 1964

Ken Kesey and the Merry Pranksters drive across America in a bus called Further, and throw the parties chronicled in Tom Wolfe's The Electric Kool-Aid Acid Test

April 1965

The Beatles are introduced to acid by George's dentist. Sgt. Pepper's Lonely Hearts Club Band comes out in June 1967, and while John denies "Lucy in the Sky with Diamonds" was an intentional expansion of "LSD", few believe him

14 January 1967

Four years after he is sacked from the psychology department at Harvard, acid evangelist Dr Timothy Leary tells a 30,000-strong gathering at the Human Be-In in San Francisco to, "Turn on, tune in, drop out"

21 February 1971