Research that aims to map the activity of serotonin in the brain could revolutionise the use of antidepressants and behavioural therapy for people with mental illnesses.

The neurotransmitter serotonin has long been associated with mood, with drugs that boost the chemical in the brain helping to alleviate the symptoms of common illnesses such as depression and anxiety, but scientists lack a deep understanding of how it mediates different mood disorders.

By understanding the biology of serotonin, the hope is that drugs can be created that only target cells relevant to a particular disorder and behavioural therapies can be made more effective, reducing the need for antidepressants.

Dr Jeremiah Cohen, an assistant professor at the Johns Hopkins Brain Science Institute in Baltimore, said: “The ultimate aim is to understand the biology of mood and how groups of cells in the brain connect to produce our emotional behaviour. Most antidepressants operate broadly in the entire serotonin system. What we hope to do with this map is use drugs that are available or design new drugs that will target only the components of that system relevant to a particular disorder.”

The use of antidepressants in England has soared since the late 1990s, raising concerns in some quarters about over-prescription. Researchers from the Nuffield Trust and the Health Foundation found that 40m prescriptions for antidepressants were made in 2012, compared to 15m in 1998. Doctors write prescriptions for more than one in 10 adults in developed countries, with Iceland, Australia, Canada and European Nordic countries leading the way, according to 2013 data from the Organisation for Economic Co-operation and Development. More than 10% of American adults have used antidepressants.

Antidepressants that are better targeted could also avoid some of their common side-effects such as insomnia and sexual dysfunction, because the drugs could be designed to only affect the part of the brain involved in mood, said Cohen.

He is one of four researchers chosen by the charity MQ: Transforming Mental Health to share £900,000 to carry out mental health research. The effectiveness of selective serotonin reuptake inhibitors (SSRIs) as antidepressants strongly suggests that serotonin transmitter pathways are involved in illnesses such as depression and anxiety but little is known about the biology of the chemical. “Almost by accident scientists discovered that drugs that work on serotonin can affect mood,” said Cohen. “We are working with a blunt system and we need to refine it.”

He will build on preliminary studies of serotonin neurons in mice while the animals perform “reward and punishment” tasks. By monitoring their behaviour during the tasks, he and his colleagues will be able to map how the neurons participate in well-known responses that are analogous to human behaviours. He said that scientists have long been interested in mapping mood in the brain by dissecting the behaviour of serotonin neurons but in the past they have not had the technology to do it. “In neuroscience we are where physics was with Galileo and Newton,” he said. “It’s basic stuff. An observer might say we should know that [already], it seems like such an obvious thing we want to understand.”

MQ chief executive Cynthia Joyce said: “Whether medication or psychological therapy, it is vital that people receive the most effective mental healthcare that works for them. Dr Cohen’s research addresses a long-standing gap in our understanding of mental illness. Excitingly, it has the potential to help us to achieve better, more personalised treatments for millions of people.”