Serotonin Concentrations: Estimation Results

We followed a supervised-learning approach for extracting serotonin signals (Kishida et al, 2016). We created a training set of voltammograms taken from a set of probes constructed identically to those used in the patient recordings. The probes were exposed to analytes in a flow cell containing a known range of serotonin concentrations (from 0 to 8000 nM) confounded with various concentrations of dopamine and pH levels. The concentration dependence of the shape and magnitude of the serotonin redox current was directly apparent (Figure 1a). We therefore trained a multivariate penalized regression model (Tibshirani, 1996) to extract serotonin concentration estimates from all points of each trace (1000 samples over 10 ms, Figure 1a). We aimed to produce a dual-transmitter model that could predict both serotonin and dopamine, as we had previously observed task-related dopamine fluctuations in these samples.

Figure 1 Serotonin concentration prediction from dual transmitter model. (a) An illustration of voltammograms acquired for varying levels of serotonin concentration (left) and dopamine concentration (right) in the flow cell. We see that low to high concentration levels produce changes in current magnitude around the oxidation potentials (insets). Concentration is denoted by []. (b) The flow cell predictions are illustrated for serotonin under varying concentrations of dopamine in the mixture. Serotonin was sampled at each level of concentration from 0.1 to 8 μM in 0.1 μM increments. For each of these 80 concentrations we computed the serotonin prediction over different levels of dopamine. Not all concentration mixtures for dopamine and serotonin were acquired and acquired mixtures are denoted by the asterisk. Over this test grid we interpolated (using linear triangulation) across the acquired tests samples to produce a three-dimensional heat map of serotonin predictions. This plot show that serotonin predictions do not vary systematically with increasing dopamine levels. We illustrate one outlying serotonin prediction, which was observed but not included in the interpolated plot for visualization purposes. (c) Flow cell predictions for dopamine in the mixture, plotted as a function of increasing dopamine and increasing serotonin as per b. Again, dopamine predictions using our model do not appear to be systematically affected by the level of serotonin in the sample. (d) We tested 200 out-of-sample voltammograms for each concentration level to quantify the error in generalizability. Illustrated in gray are predictions for different concentration ranges and their 99.99% confidence intervals. Red crosses denote the mean of the (correct) test range. For each range we randomly selected 20 predictions from the full test set. PowerPoint slide Full size image

We used L1-penalized regression to create a generalizable dual-transmitter regression model that estimated the concentrations of DA independent of the ambient levels of 5-HT and pH, and 5-HT, independent of the level of DA and pH (Supplementary Figure S1). We found that regression coefficients were distributed throughout the trace indicating that concentration levels were best predicted by considering not only the peaks of oxidation and reduction but also other points distributed across the voltage sweep (Figure 1a). Crucially, we were able to estimate the concentrations of serotonin in each mixture independent of dopamine (Figure 1b) and we could predict dopamine concentration levels independent of serotonin (Figure 1c) Moreover, our predictions for serotonin were trained to ignore altering pH levels (Supplementary Figure S2). This model estimated the true serotonin levels within 90% confidence intervals of the estimated levels in absolute terms (Figure 1d), even in the presence of dopamine and at differing pH. We also tested whether at very low concentrations we could differentiate 5-HT concentrations and found a resolution of ~100 nM (Supplementary Figure S2).

Figure 1a shows that determining the low contamination of the models is difficult to observe by visual inspection, as the voltammograms for changing dopamine concentrations appear similar to those for serotonin (Figure 1a). For additional validation of our procedure, we compared the dopamine predictions to our previously published findings (Figure 2). We confirmed that on the identical data sets to those previously published (17 recordings in total), we could replicate previous results on transient fluctuations in dopamine from the dorsal striatum (Kishida et al, 2016) (Figure 2). We also show in a supplemental analysis (Supplementary Figure S2), the correlation structure amongst our DA and 5 HT estimates, where small positive correlations were found to exist.

Figure 2 Dopamine replication in mixture model. We performed an in vivo validation by replicating the previous dopamine findings using our new multivariate mixture model. We split trials into low (0–50%), medium (60–80%), and high (90–100%) bets and examined dopamine transients at these different bet levels in response to positive and negative reward prediction errors. As per the findings reported previously for a univariate model (Kishida et al, 2016) dopamine estimates from the dual transmitter model predict dopamine encoding of prediction errors. Using two-way ANOVAs with factors RPE (negative RPE and positive RPE) and bet levels (high, medium, low) we found a significant interaction at 200 (p=0.005), 300 (p=0.0001), 400 (p=0.0016), 500 (p=0.0079), and 600 ms (p=0.02). Post hoc two-sample t-test results for each bet level and time point are illustrated (*p<0.05, ***p⩽0.001). For validation purposes, we report here, those measurements from the original cohort of 17 patients reported in Kishida et al, 2016. Data were baseline corrected to zero at 0 ms, and bar graphs depict the mean and SEM. PowerPoint slide Full size image

Behavior on the Sequential Investment Task

Figure 3a shows the sequence of events in the investment game (Lohrenz et al, 2007), which participants played during voltammetric recordings from dorsal striatum. The game was designed to elicit prediction-, prediction error-, reward-, and future investment-related signals associated with revelation of market price movements on 120 separate trials over 6 historical markets (20 moves per market). On each trial participants chose a level of investment for their current endowment with possible choices from 0 to 100% with 10% increments (Figure 3a) and submitted their choices. Then participants were shown the market move (its change in value, Figure 3a) to end a trial. Our behavioral data showed that over all subjects, bets were distributed bimodally across these 11 possible investment choices (Figure 3b), with investment levels distributed around 50% and also peaking at 100%. RPEs measure the difference between the return on a trial and a prediction. We defined the return as the fractional change in wealth (combining the current bet size and market change), and the prediction as an average of recent previous returns. We also scaled this difference by the SD of those previous returns (see Supplementary Materials and Methods). Across the cohort, this led to a spectrum of positive and negative RPEs (Figure 3c). Further, based on our previous results (Kishida et al, 2016) we considered counterfactual as well as real outcomes depending on the current betting level. This study suggested that negative outcomes could be experienced in two ways: the first were those outcomes where negative RPEs were experienced and so events were ‘worse than expected’ (as bets were high). The second were counterfactual negative events, in which positive RPEs occurred when bets were low and thus regret on a foregone gain. We correlated the whole collection of events and choices with relative fluctuations in transmitter concentrations.

Figure 3 Investment game and distributions of bets and RPEs over trials. (a) In this figure we provide an illustration of the overall task design. To investigate the role of serotonin we used an investment game (Lohrenz et al, 2007) where participants were endowed with an initial 100 ‘points’ and were instructed to invest a percentage of this amount for investment into a stock market (historic markets, eg, the 1929 Wall St crash). Participants could choose to invest 0–100% (color bar) in 10% increments (blue arrow, Bet(t)). On each trial participants submitted their investment (upper panel) and 840 ms later (±12 ms std) were shown the market return (middle panel). On the outcome, participants either lost or gained in accordance with their investment. From these market moves we calculated the reward prediction error on that trial. Following this outcome, participants submitted their next investment (blue arrow, Bet(t+1)) at their own pace (lower panel). (b) Distribution of investment choices over all participants. (c) Distribution of reward prediction errors, calculated over each market move over all participants. PowerPoint slide Full size image

Serotonin Encodes Loss Prediction Errors

We assessed serotonin responses in voltammograms at a repetition frequency of 10 Hz using the penalized regression models developed above. We examined fluctuations in estimated concentrations at the time of trial outcomes (as the market move is revealed) and tested for the serotonergic encoding of prediction errors. Figure 4a displays the serotonin transients associated with positive and negative RPEs. Remarkably, when considering all betting levels, serotonin displayed an upward fluctuation to negative prediction errors and a downward fluctuation to positive prediction errors (Figure 4a and see also Supplementary Figure S3). Given the potential difference in response to negative RPEs at high and low betting levels (loss and counterfactual loss/foregone gain, respectively), we examined serotonin fluctuations across a median split of bet levels. Figure 4b shows that this encoding reversed for the lower half of bets with upward serotonin fluctuations encoding positive errors and downward fluctuations encoding negative errors. The inversion of the encoding can be understood as the presence of a counterfactual term for serotonin, which responds to negative outcomes both in the context of a surprising loss when one was highly invested in the market and a surprising gain when one was not. The difference between Figure 4b and Supplementary Figure S3 is the baseline normalization of the signals to 100 ms before revelation of the outcome (Figure 4b) or at the time of revelation (Supplementary Figure S3). We present both as a form of exploratory result. They suggest that the dynamics of how the prediction component of the prediction error is represented in serotonin concentration would be worth exploring in future studies; in particular, higher temporal resolution in the voltammetric signal could elucidate early dynamics that alter baseline properties (Schmidt et al, 2013,Supplementary Figure S3). Here the interaction of RPE and bet amount was significant for the earlier baseline (Figure 4b). In our Supplementary Information (Supplementary Figure S3) we also include a random effects analysis across patients to ensure that our results are not driven by only a few subjects. These additional analyses support our findings when accounting for individual differences.

Figure 4 Serotonin encodes negative reward prediction errors at high bets. (a) Testing across all outcomes and separating according to either a concomitant positive or negative reward prediction error, we found that serotonin fluctuated significantly more positively for negative (black line) compared to positive (cyan line) reward prediction errors. Six two-sample t-tests were performed over temporal bins (100–600 ms) comparing concentration levels; significant effects of RPE were observed at 300 and 400 ms (*p<0.05). Here we baseline corrected at −100 ms. (b) Two-way analyses of variance of serotonin’s transient response at presentation of the outcome or market move were performed for six temporal bins (100–600 ms), with factors reward prediction error polarity; positive and negative and bet level; low (0–50%), and high (60–100%). These revealed a significant interaction of reward prediction error and bet level at 100 (F=14.34, p=0.0002) and 500 ms (F=4.89, p=0.027). Post hoc two-sample t-tests were performed using permutation testing to assess within bet range differences in the response to negative compared to positive reward prediction errors. For the high bet range (60–100% invested), serotonin transients were significantly greater for negative compared to positive reward prediction errors at 100 ms, p=0.001; 300 ms, p=0.011; 400 ms, p=0.005; and 500 ms, p=0.01. While for the low bet range (0–50% invested) responses were significantly greater for positive compared to negative reward prediction errors at 100 ms; p=0.016. Only the differential response at 100 ms in the high bet case survived FWE-correction p=0.005 (**p⩽0.005, *p⩽0.05, (**)FWE-corrected). We also applied one-sample, two-sided t-tests in order to investigate the effects of RPE and bet size on 5-HT responses as compared to baseline. We find that the difference is driven by significant decreases in 5-HT following positive reward prediction errors at high bets, and to negative reward prediction errors at low bets («p<0.005, <p<0.05). Bar graphs depict the mean and SEM. Comparisons of transients with an alternate baseline is presented in Supplementary Figure S3. (c) The area under the curve in b revealed a significant interaction (F=7.13, p=0.0077) of RPE and bet level with larger (in time and amplitude) negative-going transients for positive reward prediction error responses in the high bet condition. (d) We tested the serotonin response at 100 ms and its correlation with the sign and polarity of the RPE. After omitting 65 outliers (~3% of trials) that may drive the effect (outliers defined as RPEs with an absolute magnitude >3 and Z-scores with an absolute magnitude >5) we see a small but significant correlation for the different bet levels. Serotonin transients are negatively correlated with the RPE for high bets (R=−0.0714; p=0.0113) and positively correlated with the RPE for low bets (R=0.0653; p=0.0494). To explore these results more granularly, we examined individual bins. We found that the only significant individual bins were at (20 and 30%), (60 and 70%), and (80 and 90%) with correlation coefficients and p-values of (R=0.19; p=0.01), (R=−0.08; p=0.06), and (R=−0.14; p=0.009), respectively. This suggests that a putative ‘indifference point’ for counterfactual and actual losses occurs around 40–50%. PowerPoint slide Full size image

To allow for duration of serotonergic signaling to be altered in response to RPEs, we computed the area under the curve to indicate ‘cumulative serotonin’ responses. For this analysis the interaction of prediction error (positive or negative) and bet invested (high or low) was significant (Figure 4c). In particular, the response to positive RPEs seemed to induce a depression in serotonin that was more prolonged than in the low bet condition. Further, a parametric analysis revealed a small but significant negative correlation between the serotonin response and RPE at high bets and a small but significant positive correlation between serotonin response and RPE at low bets (Figure 4d). To examine the subjective effects of actual and counterfactual gains and losses, rather than to RPE per se, we conducted a further supplemental analysis (Supplementary Figure S4). This revealed a lack of a parametric effect in gains or losses (Supplementary Figure S4).

Serotonin Protects Investors from Loss

Given these bet-dependent prediction error transients, we sought to establish serotonin’s influence on investment decisions. The effect of counterfactual outcomes on both dopamine (Kishida et al, 2016) and serotonin (Figure 4) suggests that it is crucial to perform the analysis of action encoding (betting more or less) at different bet levels as a bet of 0% could result in large foregone gains (ie, counterfactual losses), while a bet of 100% could result in large actual losses. In other words, in the context of this task, one’s next move carries two distinct risks of loss on the upcoming trial. We tested whether fluctuations in 5-HT could be used to predict the bet level on the next trial. Using a multiple linear regression we tested for serotonin and game factors in predicting the next decision. Specifically, our independent variables included the area under the curve of the 5-HT transient from 100 to 600 ms, the bet level, the polarity of the RPE at trial (t), as well as their interactions. Our dependent variable was the change in bet at trial (t+1). Our regression model revealed significant predictive power in upcoming decision (F-statistic vs constant model: 32.4, p-value <0.00001; Supplementary Table 2). Importantly the regressor describing the interaction of serotonin and current bet level was a significant predictor of the upcoming decision (p=0.04). This was a negative interaction indicting that for large serotonin responses and large bets, participants tended to decrease their bet, and for large serotonin responses and small current bets, participants tended to increase their bets. We found that the three-way interaction of serotonin, bet level, and RPE sign was at trend level significance (p=0.13; Supplementary Table 2).

To examine and illustrate these regression effects, we first separated out ranges of current bet levels (Figure 5a), and examined how serotonin transients were associated with decisions following a negative RPE (Figure 5). We tested the relationship between serotonin responses and current bet levels at trial (t) for decisions to ‘lower’ and for decisions to ‘hold or raise’ the bet on trial (t+1) (Figure 5b). These analyses are a recapitulation of the negative RPE responses in Figure 4 but separated according to what the subject decides to do next. We found that under the conditions of the decision to withdraw from the market following negative RPEs there was a strong positive correlation between 5-HT and current betting levels (Figure 5c). This is important given that withdrawal from the market (ie, lowering one’s bet) is consistent with the hypothesized role for serotonin in forms of avoidance (Dayan and Huys, 2008). This striking parametric effect is indicated in the serotonin time courses of Figure 5b. We can see that reducing the bet from a high amount implies reducing the risk of actual loss, and is associated with positive serotonin fluctuations (Figure 5b). Reducing the bet from an already low amount implies increasing the risk of counterfactual losses, and is associated with negative fluctuations. A trend toward a significant positive correlation for serotonin and decisions to hold or raise one’s bets was also observed. In the time courses we can see that particularly at 10–20% bet levels, serotonin rises following a negative RPE and is associated with a subsequent raise-or-hold bet decision. This direction is again consistent with serotonin protecting against counterfactual losses on the next trial.

Figure 5 Serotonin and active avoidance following negative reward prediction errors. (a) Depiction of ‘next actions’. Responses to trial (t) were analyzed for bet(t)=(0%, (10–20%), (30–40%), (50–60%), (70–80%), and (90–100%)). For each of these six levels we examined ‘lower bet’ next actions (black arrows) and ‘hold-or-raise’ bet next actions (gray arrows). (b) The n=842 negative RPE transients presented in Figure 4a and b are represented here but separated according to next-bet decision and current betting level. These results are provided in order to explore the significant negative interaction between serotonin and current bet on predicting change in bet (Supplementary Table 2). Consistent with this negative interaction in the regression analysis, we observe that large positive 5-HT transients at large bets predict a lowering of the bet at trial t+1 (black line, right panels). While dips in 5-HT are associated with reducing one’s bet at low bet levels to even lower levels (black line, left panels). The opposite effect is observed for holding or raising one bets, grey lines (with 0% not showing any significant transient effects). Significance here is indicated for uncorrected t-tests against zero (*p<0.05, **p<0.01, ***p<0.005). (c) Applying a correlation analyses to examine the relationship between serotonin and current bet levels when the next decision is to lower one’s bet. We find that over all ‘lower bet’ decisions, serotonin, and the current bet level were positively correlated (R=0.3; p<0.00001). This indicates that market withdrawal is affected by serotonin following poor outcomes. More specifically it indicates that serotonin may prevent further withdrawal (when investment is already low) and promote withdrawal (when investment is high). (d) Correlating decisions to hold or raise bets suggests the opposite effect (but at only trend-level significance). PowerPoint slide Full size image

In Figure 6 we explore the same dependencies but following positive RPEs. Here at low bets, again from 10 to 20% levels we observe a upgoing serotonin transient (Figure 6b) that dominates the low bet regime (Figure 4b). At this betting level both ‘lower’ and ‘raise-hold’ decisions are associated with a positive transient. Significant negative-going fluctuations are observed at the lowest betting level of 0%. No parametric effects are observed for either decision following positive RPEs (Figure 6c and d).

Figure 6 Serotonin and active avoidance following positive reward prediction errors. (a) Depiction of ‘next actions’ as per negative RPE analysis in Figure 5. (b) Serotonin transients (n=882) following positive reward prediction errors as per Figure 4a and b but shown here separated according to decision on next trial and current bet level (lower bet on trial (t+1): cyan line; hold-or-raise bet on trial (t+1): dark green line). Only at low bets (where counterfactual losses dominate) did we observe large transients—the direction of the transient was not discriminative however in terms of next-bet decision. (c) Unlike following negative RPES, no parametric effect, in terms of current bet level and serotonin response, was observed for the decision to lower bet following positive reward prediction errors. (d) Similarly, no parametric effects were observed for serotonin responses preceding a decision to raise or increase bet levels (following a positive reward prediction error). PowerPoint slide Full size image

In order to investigate the timing of these decision-related transients further, we extracted the peak 5-HT response from every trial and tested whether a faster time to peak corresponded with decisions from investment on the next trial. For responses to negative RPEs we found no timing effects in an analysis of decision × bet level. However, in response to positive RPEs we saw a significant effect of time to peak on the decision to lower, hold, or raise one’s bets following the outcomes. Specifically, the decision to raise one’s bets was associated with slower 5-HT transient peaks as compared to decisions to hold or reduce current betting levels. No effect of bet level or interaction was observed (Supplementary Figure S5). This may suggest that fast serotonin signals are associated with withdrawal from market investment, even when the ‘going is good’.