An interdisciplinary team of researchers from the University of Texas Medical Branch at Galveston and the University of Houston has found a new way to influence the vital serotonin signaling system—possibly leading to more effective medications with fewer side effects.

Scientists have linked malfunctions in serotonin signaling to a wide range of health issues, everything from depression and addictions to epilepsy and obesity and eating disorders. Much of their attention has focused on complex proteins called serotonin receptors, which are located in the cell membrane. Each receptor has a so-called "active site" specially suited to bond with a serotonin molecule; when that bond is formed, the receptor changes shape, transmitting a signal to the cell's interior.

Traditional drug discovery efforts target interactions that take place at such active sites. But a receptor's behavior can also be changed by additional proteins that bind to the receptor at locations quite distant (in molecular terms) from the active site, in a process called "allosteric regulation"—the mechanism examined by the UTMB-UH team for one specific and highly significant kind of serotonin receptor, designated the 5-HT2C.

"This is a whole new way of thinking about this system, targeting these interactions," said UTMB professor Kathryn Cunningham, senior author of a paper on the research now online in the Journal of Neuroscience. "Basically, we've created a new series of molecules and validated that we can use them to change the way the receptor functions both in vitro and in vivo, through an allosteric effect."

The UTMB-UH group's approach centers on the natural interaction between the 5-HT2C receptor, serotonin, and another molecule called PTEN. Like serotonin, PTEN controls 5-HT2C receptor function; because it does so at a location distant from the active site, it's possible—and in fact common—for a receptor to bind to serotonin and PTEN simultaneously. When this happens, an allosteric effect is produced: serotonin signaling is weakened.

"We want to maintain signaling through 5-HT2C receptors to gain therapeutic benefits, and to do that we had to reduce the number of receptors that were binding to PTEN molecules," said UH professor Scott Gilbertson, another senior author on the paper. "One way to do that is to develop an inhibitor that competes with the receptor for binding to PTEN."

The candidate inhibitor chosen by the researchers was a fragment of the receptor itself—specifically, the part of the receptor where PTEN attaches. Such sub-protein structures are known as "peptides"; this one was previously dubbed "3L4F." Test-tube and cell-culture experiments showed that 3L4F boosted 5-HT2C response significantly; that it acted by binding to PTEN; and that it had no effect at all on another type of serotonin receptor, designated 5-HT2A.

Behavioral studies in laboratory rats also indicated that 3L4F increased 5-HT2C responses.

"We looked at both human cells and rats because ultimately we want to translate this research into therapeutics," said UTMB postdoctoral fellow Noelle Anastasio, lead author of the paper. "The idea of targeting these interactions to produce drug and research tools is truly new and has great potential."

The team took a step toward realizing that potential by trimming 3L4F down to a peptide roughly half its size that retains similar efficacy. Using computational molecular modeling, they determined which elements of this peptide were important to bonding with PTEN—information they will use to design smaller molecules with the same or better activity.

"We've got the basics down now, so we can use the chemistry to make new molecules that we think might be potentially useful for treatment of addictions, for example," Cunningham said. "But there's also an intense interest in figuring out the biology of this interaction between 5-HT2C and PTEN, what it means in terms of disease states like the addictions, alcoholism, depression and obesity and eating disorders. I think in a broader sense this is really going to help us understand the neurobiology of these disorders."

Explore further Low dose of serotonin-acting chemical improves blood sugar tolerance