5-HT 1B receptors in relation to anxiety

Knockout of the 5-HT 1B receptor gene results in mice with a distinct behavioral phenotype, aggressive and less cautious (Saudou et al. 1994; Zhuang et al. 1999). Decreased anxiety in the 5-HT 1B receptor knockout mice has consequently been reported, with low anxiety in the elevated plus maze and more activity in the open field (Brunner et al. 1999; Nautiyal et al. 2016; Zhuang et al. 1999). In line with this, overexpression of 5-HT 1B receptors in the dorsal raphe nucleus has yielded increased stress-induced anxiety in rats (Clark et al. 2002). In contrast, the 5-HT 1B receptor-overexpressing rats displayed less anxiety, before exposure to stress (Clark et al. 2002). Furthermore, in nonstressed rats, there was an inverse correlation between 5-HT 1B receptor mRNA expression in dorsal raphe nucleus and anxiety, as estimated with activity in the elevated plus maze (Kaiyala et al. 2003). In serotonergic projection areas in the brain, high 5-HT 1B receptor density has been reported in a genetic mouse model for anxiety (Clement et al. 1996).

Another major source of information about the behavioral effects of 5-HT 1B receptors stems from extrapolation of the effects of 5-HT 1B receptor ligands. Most reports describe anxiogenic effects of 5-HT 1B receptor agonists (Sari 2004), with increased anxiety in paradigms as the elevated plus-maze (Lin and Parsons 2002; Solati et al. 2011). Sari hypothesized that 5-HT 1B receptors modulate anxiety through inhibition of serotonin, acetylcholine, and GABA (Sari 2004). However, anxiolytic properties of 5-HT 1B receptor agonists have also been reported. The early reports on reduced anxiety upon 5-HT 1B receptor activation have been dismissed as results of nonselective ligands (Sari 2004). Still, reduced anxiety-related behavior has been demonstrated with CP-94,253, a selective 5-HT 1B receptor agonist, in the Vogel conflict drinking test, which is sensitive to the acute anxiolytic effects of bensodiazepines, but fail to detect SSRI effect (Chojnacka-Wojcik et al. 2005). Furthermore, CP-94,253 attenuated the anxiogenic effects of cocaine (Klein et al. 2016), while the 5-HT 1B receptor antagonist increased cocaine-induced anxiety (Hoplight et al. 2005). Otherwise, in baseline conditions, the effect of 5-HT 1B receptor antagonists is more clear and consequent. Anxiolytic properties of 5-HT 1B receptor antagonists has been demonstrated in a variety of paradigms for anxiety such as Vogel drinking test (Chojnacka-Wojcik et al. 2005), separation-induced vocalization (Dawson et al. 2006; Hudzik et al. 2003; Zhang et al. 2011), human threat test (Dawson et al. 2006), and reactions to a novel environment (Nowicki et al. 2014). In humans, studies on the role of 5-HT 1B receptors in anxiety are disproportionately sparse. Increased fear of public speaking has been reported in healthy volunteers after administration of the 5-HT 1B/1D receptor agonist sumatriptan (de Rezende et al. 2013). In the single in vivo brain imaging study of subjects with posttraumatic stress disorder, low 5-HT 1B receptor binding was described in the caudate, amygdala, and anterior cingulate cortex (ACC). The significant differences in binding compared with control were driven by the low 5-HT 1B receptor binding potential (BP ND ) in a subgroup of subjects with major depressive disorder comorbidity (Murrough et al. 2011a).

Another condition in which 5-HT 1B receptors have been suggested to be involved in the etiology and pathophysiology is obsessive-compulsive disorder (OCD), which has a complex, though fundamental, relation to anxiety. In genetic studies on 5-HT 1B receptors in OCD, an association between the G861C polymorphism in the 5-HT 1B receptor gene and OCD has been found and replicated (Kim et al. 2009; Mundo et al. 2000). However, in a study of OCD probands and parents, the silent 5-HT 1B receptor gene G861C polymorphism in the coding region was not transmitted to a significantly larger degree than its counterpart, in a study population with a family history of OCD for 40% of the probands (Di Bella et al. 2002). Gender differences might contribute to inconsistent results, since Kim et al. only examined men, and a broader genetic study found significant transmission of 5-HT 1B receptor gene polymorphisms only to male and not to female probands (Mas et al. 2014). Furthermore, a recent positron emission tomography (PET) study found no differences in cerebral 5-HT 1B receptor binding in OCD patients compared with controls (Pittenger et al. 2016). Still, the 5-HT 1B/1D receptor agonist sumatriptan has been reported to aggravate OCD symptoms in subjects with relatively low OCD symptom levels (Gross-Isseroff et al. 2004), although no effect on OCD was found in a group with more OCD symptoms, when subjects were given another 5-HT 1B/1D receptor agonist, zolmitriptan, with higher blood-brain barrier penetrance (Boshuisen and den Boer 2000). With a more 5-HT 1B receptor selective compound, RU24969, with high affinity for 5-HT 1B receptors and low affinity for 5-HT 1A receptors, and simultaneous administration of the 5-HT 1A receptor antagonist WAY100635, a dose-dependent increase of locomotor stereotypy has been demonstrated in a rodent model of OCD. The RU24969-exposed rats had increased Fos expression, an indirect marker of neuronal activity, in the dorsal striatum, a region implicated in PET studies of OCD (Saxena and Rauch 2000). This RU24969-induced increase of striatal neuronal activity was obliterated with SSRI pretreatment (Ho et al. 2016). Altogether, animal data imply a role for the 5-HT 1B receptor in anxiety, though human studies are sparse and inconclusive.

5-HT 1B receptors in relation to depressive states

Depression is the psychiatric condition with most reports in the literature in relation to the 5-HT 1B receptor. A large part of previous 5-HT 1B receptor depression research stems from preclinical studies, mostly in rodents (Ruf and Bhagwagar 2009). Mice constitutionally or conditionally genetically deprived of 5-HT 1B receptors not only are less anxious, but also show less depression-like behavior, with less immobility time in both the forced swim test (FST) and the tail suspension test (TST) (Jones and Lucki 2005; Nautiyal et al. 2016), and higher sucrose preference (Bechtholt et al. 2008; Nautiyal et al. 2016), Jones and Lucki found significantly lower immobility time only in female 5-HT 1B receptor knockout mice compared to wild-type mice. Furthermore, in a number of microdialysis studies, an augmentation of serotonin levels in response to SSRI was found in the hippocampus (Knobelman et al. 2001; Malagie et al. 2001; Nautiyal et al. 2016), but not in the striatum (De Groote et al. 2003; Knobelman et al. 2001) of 5-HT 1B receptor knockout mice compared with controls. The regional difference in SSRI-induced serotonin release may be due to innervation, with hippocampus receiving serotonin input mainly from the 5-HT 1B receptor key region the median raphe nucleus, while the striatum receives projections from the dorsal raphe nucleus (Knobelman et al. 2001; Tork 1990). By contrast, the knockout for the 5-HT 1B receptor-related p11 gene has resulted in a depressive phenotype, with more immobility time and lower preference to sucrose than wild-type littermates (Svenningsson et al. 2006). On the other hand, 5-HT 1B receptor binding in the p11 knockout mice is reduced, but not depleted (Svenningsson et al. 2006). This more moderate reduction in 5-HT 1B receptor levels would be in line with human case-control studies, in which globally low brain binding and mRNA expression has been found in patients with major depressive disorder (MDD) (Tiger et al. 2016) and suicide subjects (Anisman et al. 2008), respectively. The behavioral consequences of having low versus no 5-HT 1B receptors in the brain may differ considerably.

Animal models

The results from studies of 5-HT 1B receptors in animal models for depression are largely inconclusive. Low 5-HT 1B receptor binding has been demonstrated in the hippocampus in a rat model for inherited depressive traits, Flinder’s sensitive line, and in rats separated from their mothers. The effects of either genetic or environmental vulnerability for depression on 5-HT 1B receptor binding could be reversed with antidepressants (Shrestha et al. 2014). Likewise, in Rgs2-mutant mice, with long latency to eat in the novelty suppressed feeding test as the main behavioral proxy for depressed mood, raphe nuclei 5-HT 1B receptor gene expression was low (Lifschytz et al. 2012). On the other hand, higher 5-HT 1B receptor densities in most brain regions, including dorsal hippocampus and the rostral raphe nuclei, were reported in Flinder’s sensitive line rats, both compared with Flinder’s resistant line and Sprague-Dawley rats (Nishi et al. 2009). Furthermore, an early finding in the field was the twofold higher 5-HT 1B receptor binding in the cortex, hippocampus, and septum in rats that reacted with learned helplessness in reaction to uncontrollable electric shocks versus nonhelpless rats (Edwards et al. 1991). To complicate things further, high 5-HT 1B receptor mRNA in dorsal raphe nucleus has been reported both in rats with learned helplessness (Neumaier et al. 1997) as well as in stress-resilient rats (Neumaier et al. 2002).

Effects of agonists and antagonists in animal models of depression

The role of the 5-HT 1B receptor in depression-like states is more clearly disentangled in studies with drugs targeting the receptor. Blocking the inhibitory 5-HT 1B receptor would in theory lead to increased levels of extracellular serotonin, and potential antidepressant effects. Indeed, treatment with the 5-HT 1B/1D receptor antagonist GR 127935 increased latency to immobility in guinea pigs in FST (Rex et al. 2008). However, in most studies of 5-HT 1B receptor antagonism in depression-like states, effect has been demonstrated only with simultaneous administration of antidepressant drugs (Ruf and Bhagwagar 2009). 5-HT 1B receptor antagonists have been found to increase the anti-immobility effect in FST when coadministered with tricyclic antidepressants or a monoamine oxidase inhibitor (Chenu et al. 2008; Tatarczynska et al. 2004a) or with an SSRI (Tatarczynska et al. 2002). However, the 5-HT 1B/1D receptor antagonist GR 127935 has also been reported to block the effects of paroxetine and imipramine in mice in the tail suspension test (O'Neill et al. 1996), whereas in the same study, depression-like immobility was decreased and the antidepressant effect of imipramine was increased with the 5-HT 1B receptor agonist RU 24969 (O'Neill et al. 1996). Furthermore, reduced depression-like behavior has been demonstrated with different 5-HT 1B receptor agonists also in the forced swimming test (Chenu et al. 2008; Tatarczynska et al. 2004b). The findings that antidepressant-like effects can be obtained with both antagonists and agonists targeting the receptor likely reflect the heterogeneous localization of 5-HT 1B receptors in different neuronal populations where autoreceptors and heteroreceptors may differently modulate depression-like behaviors (Chenu et al. 2008; Nautiyal et al. 2016).

Human studies of 5-HT 1B receptors in MDD

Even though preclinical studies have had immense importance for the literature on 5-HT 1B receptors in depression, translation of animal data to the clinical diagnosis MDD poses several challenges. Although animal models of depression are sensitive to antidepressants, none of them reflect the episodic feature of MDD and the most ominous MDD symptom, suicidal ideation, is absent in other species than humans. While some animal models may mimic vulnerability to stress and thereby the characteristic disproportionate load of symptoms such as lowered mood in MDD, no model convincingly display signs of the symptoms that may persist also without precipitating external causes (Belmaker and Agam 2008). That said, it is also challenging to validly study the pathophysiology of major depressive disorder also in clinical studies, mainly due to difficulties in defining the diagnosis. The main shortcomings of the MDD definitions in the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM), the most widely used MDD definition, as well as in ICD-10, is the heterogeneity of the current MDD diagnosis, which can be fulfilled in 1497 different ways (Ostergaard et al. 2011), and the disregard of current psychosocial situation. Evaluating depressive symptoms in relation to current context is most probably necessary to avoid erroneously labeling stressed subjects as depressed (Horwitz and Wakefield 2007). These flaws in the current definition of MDD may have hampered the research on pathophysiology in MDD and may explain some of the inconsistent results. Still, in the emerging picture of accumulating 5-HT 1B receptor MDD data, the 5-HT 1B serotonin receptor subtype seems relatively consistently associated with or altered in this disorder. In a study of 394 psychiatric patients, an association between the silent G861C 5-HT 1B receptor polymorphism, which is associated with low 5-HT 1B receptor brain density (Huang et al. 1999), and a history of MDD or substance abuse disorder was found (Huang et al. 2003). Moreover, there is indirect neuroendocrine evidence for altered 5-HT 1B receptor function in MDD, with blunted growth hormone (GH) response to the 5-HT 1B/1D receptor agonist sumatriptan in MDD patients (Cleare et al. 1998) and likewise reduced GH response to another triptan, zolmitriptan, which passes the blood brain barrier to a higher degree than sumatriptan, in a subpopulation of MDD patients with melancholic depression (Whale et al. 2001).

Comparisons of 5-HT 1B receptor measurements in subjects with MDD with those of controls are fundamental for knowledge about this receptor in the pathophysiology of MDD. At present, information regarding 5-HT 1B receptor-related measures are available from studies of mRNA expression postmortem in forebrain regions (Anisman et al. 2008; Lopez-Figueroa et al. 2004) as well as PET studies of radioligand binding in vivo (Murrough et al. 2011b; Tiger et al. 2016).

PET offers molecular imaging of the living human brain at high anatomical resolution (Farde 1996). Brain PET requires a lipophilic ligand that can be labeled with a radioactive isotope and bind selectively to a target protein such as the 5-HT 1B receptor (Halldin et al. 2001). Tracer doses of the radioligand are injected intravenously, and a small portion passes the blood-brain barrier and bind to the target protein. The radioligand emits positrons that after a short distance form particle pairs with electrons. The particle pairs are annihilated and transformed into photons that traverse the skull and subsequently are detected by the surrounding PET system, enabling quantification of target protein binding in vivo in the brain. At present, there are two radioligands available for 5-HT 1B receptor imaging in humans: [11C]P943 and [11C]AZ10419369 (Zimmer and Le Bars 2013). Both [11C]AZ10419369 and [11C]P943 have high affinity for 5-HT 1B receptors (K D = 0.4 and K D = 1.2 nM, respectively) (Paterson et al. 2013).

Despite the methodological differences and the small number of cases and controls, previous postmortem and in vivo studies consistently support a trend of low levels of cerebral 5-HT 1B receptor related measures in unmedicated MDD subjects (Anisman et al. 2008; Murrough et al. 2011b; Tiger et al. 2016).

Interestingly, the pattern of low 5-HT 1B receptors appears to be most prominent in the limbic cortices, in regions of reported relevance for MDD, such as the anterior cingulate cortex and hippocampus (Mayberg 1997; Steele et al. 2007). Low 5-HT 1B receptor binding has been measured with PET in the anterior cingulate cortex (ACC), both in patients with recurrent MDD (Tiger et al. 2016) and in patients with MDD and posttraumatic stress syndrome comorbidity (Murrough et al. 2011a) (Table 1). Moreover, lower p11 mRNA and protein levels in MDD patients compared with controls have been found postmortem in ACC, a region with distinct coexpression of p11 and 5-HT 1B receptors (Egeland et al. 2011). However, in a recent [3H]AZ10419369 postmortem study allowing antidepressant medication MDD patients did not differ from controls in 5-HT 1B receptor binding in pregenual ACC (Veldman et al. 2017). The absence of difference in 5-HT 1B receptor binding in MDD in this autoradiography study could possibly be explained by pharmacological increase in 5-HT 1B receptor BP ND , as has been demonstrated in cortical regions after SSRI in healthy subjects (Nord et al. 2013), canceling out any low 5-HT 1B receptor binding in MDD patients. Furthermore, despite antidepressant treatment in 10 out of 12 patients, the MDD group stood out with low 5-HT 1B receptor binding in females compared with males, in contrast with the control group and patients with schizophrenia and bipolar disorder, respectively (Veldman et al. 2017).

Table 1 PET studies of 5-HT 1B receptor binding in MDD Full size table

Likewise, in the hippocampus, 5-HT 1B receptor binding was low in recurrent MDD as measured in vivo with PET (Tiger et al. 2016), and in patients with comorbidity of PTSD and MDD (Murrough et al. 2011a), and mRNA expression was numerically lower in parts of the hippocampus in MDD detected postmortem with in situ hybridization analysis (Lopez-Figueroa et al. 2004). Furthermore, in MDD subjects who had committed suicide, lower hippocampal 5-HT 1B receptor (in female subjects) and p11 mRNA expression compared with controls was found (Anisman et al. 2008). In the same study, low 5-HT 1B receptor and p11 mRNA expression was found in the frontopolar cortex. In the hypothalamus, by contrast, 5-HT 1B receptor mRNA expression was high in MDD suicide subjects (Anisman et al. 2008).

In the prefrontal cortex, 5-HT 1B receptor binding results so far have been negative, with no effect of MDD, neither in vivo with PET (Tiger et al. 2016) nor in membrane homogenates postmortem (Huang et al. 1999). Furthermore, in the basal ganglia, inconsistent findings have been reported in two PET studies. While Murrough et al. described low 5-HT 1B receptor binding in the ventral striatum/ventral pallidum in MDD (Murrough et al. 2011b), we could not replicate this difference between MDD patients and controls (Tiger et al. 2016). However, the MDD populations in the two studies differed in a number of aspects, with the first study including subjects that were smoking, subjects with higher BMI, and a larger proportion of subjects with no previous exposure to antidepressant drugs, in contrast with the subjects of the latter study. There were also technical differences in the two studies, such as the use of different 5-HT 1B receptor ligands, [11C]P943 (Murrough et al. 2011b) and [11C]AZ10419369 (Tiger et al. 2016), respectively, and different approaches in defining regions of interest (ROI), where Murrough et al. used a combined ROI from an automated template and we defined the ventral striatum and pallidum manually as separate ROIs. Perhaps most importantly, with the limited number of subjects in the two studies, there is a risk of type II errors, with failure to detect differences between MDD patients and controls due to low power.

In conclusion, low 5-HT 1B receptor binding and expression in ACC and hippocampus, key regions of the neurocircuitry of MDD (Mayberg 1997) has been reported.

5-HT 1B receptors and substance abuse

Loss of control is a core feature in substance abuse disorders. Loss of 5-HT 1B receptors in rodents induces a phenotype with reduced impulse control (Bouwknecht et al. 2001; Nautiyal et al. 2015). The involvement of 5-HT 1B receptors in abuse has mainly been studied in relation to alcohol and cocaine, though there are also reports relating to amphetamine effects (Miszkiel et al. 2011) and pathological gambling (Potenza et al. 2013).

Alcohol consumption has been reported to be high in 5-HT 1B receptor knockout mice (Crabbe et al. 1996). Correspondingly, low 5-HT 1B receptor densities have been demonstrated with autoradiography in cingulate and retrosplenial cortices, septum, and amygdala in alcohol-preferring rats (McBride et al. 1997). Furthermore, 5-HT 1B receptor agonists reduce alcohol self-administration in rats (Maurel et al. 1999; Tomkins and O'Neill 2000). On the other hand, overexpression of 5-HT 1B receptors in the nucleus accumbens in rats lead to increased alcohol consumption (Hoplight et al. 2006). In humans, associations between the 5-HT 1B receptor polymorphisms A161T in the 5′ regulatory region, with high reporter gene expression, and G681C and alcohol dependence (Sun et al. 2002) and antisocial alcoholism (Lappalainen et al. 1998), respectively, have been found. Additionally, high 5-HT 1B receptor binding in the ventral striatum in subjects with alcohol dependence has been demonstrated with PET (Hu et al. 2010). However, no differences between alcoholic and control subjects in 5-HT 1B receptor density in nucleus accumbens or any other examined brain region were found postmortem (Storvik et al. 2012). Furthermore, Huang et al. found no association between the 5-HT 1B receptor gene polymorphism G861C and alcoholism (Huang et al. 2003). Thus, despite neatly consistent small animal model data, the role of 5-HT 1B receptors in alcohol abuse remains unclear. This underscores the necessity of translational validation of animal model data in humans as a mandatory part of disease research.

Regarding 5-HT 1B receptors and cocaine, mice lacking the receptor self-administer cocaine to a higher degree (Castanon et al. 2000; Rocha et al. 1998). However, inhibition of 5-HT 1B receptors has either reduced or had no effect on cocaine self-administration (Miszkiel et al. 2011). When activating 5-HT 1B receptors, the acquired effect on the reinforcing effects of cocaine has been reported to depend on the current state in relation to cocaine intake. 5-HT 1B receptor agonists and 5-HT 1B receptor gene transfer into the nucleus accumbens increase cocaine reinforcement during a self-administration period and decrease cocaine seeking and reinstatement of cocaine seeking behavior when given during abstinence (Neisewander et al. 2014; Pentkowski et al. 2012). Moreover, viral overexpression of 5-HT 1B receptors in nucleus accumbens reduces cocaine-seeking behavior elicited by cocaine priming (Nair et al. 2013). The 5-HT 1B receptor effect on cocaine reinforcement may relate to cocaine-induced receptor trafficking, as 5-HT 1B receptor mRNA increase in the dorsal striatum upon initiation of cocaine administration and decrease during abstinence (Neumaier et al. 2009). In humans, low 5-HT 1B receptor binding in vivo has indeed been described during abstinence not in striatum but in regions connected to the ventral striatum: anterior cingulate cortex, hypothalamus, and frontal cortex, in subjects with cocaine dependence (Matuskey et al. 2014).

5-HT 1B receptors and aggression