Smell in mammals turns out to be more complex than we thought. Rather than one receptor family exclusively dedicated to detecting odors, a study in mice reports that a group of neurons surrounding the olfactory bulb use an alternative mechanism for catching scents. These "necklace" neurons, as they're called, use this newly discovered olfactory detection system to respond to odors that elicit instinctive responses, such as pheromones and the smell of seeds and nuts. Harvard researchers report the finding May 26 in Cell.

"Our work suggests that mammalian mechanisms for smell are not monolithic in terms of mechanism or logic, but rather can take many forms and can be mediated by multiple types of receptors," says senior author Sandeep Robert Datta, a neurobiologist at Harvard Medical School. "These findings revise our canonical view of how animals probe the chemical environment."

Nobel Prize-winning work back in 1991 showed that, in mammals, each sensory neuron in the main olfactory system expresses one type of G-protein coupled receptor (GPCR), which is specialized to detect a specific type of odor. The pattern of activity of all sensory neurons in the olfactory system allows us to distinguish between different odors present in the environment. This one-GPCR-per-neuron pattern also exists in the vomeronasal olfactory system, which is specialized for recognizing pheromones, suggesting a common and general logic for processing smell. Yet, a third olfactory system consisting of necklace neurons, so-called due to the unique circular pattern of their projections to the brain, also responds to diverse odors. It has not been clear which receptors are expressed by these neurons and what role they play in odor perception.

In the new study, Datta and his team discovered that necklace neurons in mice do not express GPCRs, unlike all other types of olfactory sensory neurons in mammals. Rather, these neurons express the MS4A class of proteins, which were previously not known to play a role in odor perception. Moreover, each necklace neuron expresses multiple types of MS4A receptors, in stark contrast to the one-receptor-per-neuron rule that organizes insect and other mammalian olfactory systems. These receptors respond to fatty acids that are specifically found in nuts and seeds, as well as a pheromone known to be aversive to mice.

"This discovery strongly suggests that the brain must be interpreting information from these receptors using a very different strategy from the one used by the brain to discriminate most odors," Datta says. "We speculate--but don't have the evidence to back this idea yet--that the MS4As are used as a kind of alert system for the brain, letting it know that something of real importance is out there in the world, but not telling the brain explicitly what that thing is."

By analyzing differences between Ms4a genes across mammalian species, the researchers found that the evolution of these genes preceded the advent of the mammalian receptors for taste and for pheromones. "The fact that the MS4As have been preserved for at least 400 million years suggests that these receptors play a crucial role in enabling animals to interact with the olfactory environment," Datta says.

In humans, MS4A receptors have previously been found in the intestines, lung cells, and even sperm cells. Based on the pattern of expression of MS4A receptors in different tissues, and the type of odors they detect, Datta suspects that MS4A molecules represent an ancient mechanism for sensing ethologically salient small molecules in the environment. "It is possible that this is the main function of the MS4As across species, and that the olfactory function of the MS4As is actually more recently evolved," Datta says.

For now, it is not clear whether MS4As in humans serve as odor receptors. In future studies, Datta and his team will examine whether MS4A proteins act as a primordial odor receptor across species. "This would be incredibly interesting, as it would suggest that many animals have a kind of hidden nose we were unaware of buried within their main olfactory system," Datta says.

Because the MS4A proteins are expressed on many cells in the body, the researchers will also test whether they detect cues that are generated by the body itself. "If so, this would suggest that one reason that the Ms4As are so widespread across evolution is that they are generally well suited to the detection of small molecules in the environment, regardless of whether that environment is the external or the internal world," Datta says.