Classification of Resting-State Brain Activity

We applied the multivariate models of emotional experience to brain activation acquired from young adults during resting-state fMRI (n = 499; Fig 2A). Two consecutive runs of resting-state scans were acquired, spanning a total duration of 8.53 min. Following preprocessing of data, we computed the scalar product of the resting-state signal and emotion category-specific model weights at every time point of data acquisition. This procedure yielded scores that reflect the relative evidence for each of seven emotional states across the full scanning period. A confirmatory analysis revealed that voxels distributed across the whole brain informed this prediction, as opposed to activity in a small number of brain regions (S1 Fig).

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Fig 2. Emotional states emerge spontaneously during resting-state scans. (A) Procedure for classification of resting-state data. Scores are computed by taking the scalar product of preprocessed data and regression weights from decoding models. (B) Frequency distributions for the classification of all seven emotional states (n = 499). The mean, 25th, and 75th percentiles are indicated by black lines. The solid gray line indicates the number of trials that would occur from random guessing. The data underlying this figure can be found in S1 Data. The raw fMRI resting state data can be obtained from https://www.haririlab.com/projects. https://doi.org/10.1371/journal.pbio.2000106.g002

If emotional brain states occur spontaneously, the frequency of classifications from our decoding models should be more varied than the uniform distribution that would be expected by chance. To test this hypothesis, we sought to identify whether the total time (or absolute frequency) in each state differed across emotion categories. Such an analysis informs the degree to which discrete emotional brain states may spontaneously occur and, by extension, could contribute to the identification of individual differences that map onto the likelihood of experiencing specific spontaneous states. To perform this comparison, we identified the single model with the maximum score at each time point (one-versus-all classification) and summed the number of time points assigned to each category. The frequency of emotional states clearly differed across categories (Fig 2B, χ2 = 1491.52, P < .0001, Friedman test), in contrast to the uniform distribution that would be expected if emotional brain-states did not occur in spontaneous activity (see S2 Fig).

Follow-up comparisons revealed that neutral states occurred more frequently than chance rates (20.1 ± 3.59% [s.d.], z = 20.50, P unc = 2.03E-93), followed by states of surprise (18.37 ± 3.87% [s.d.], z = 16.38, P unc = 2.47E-60) and amusement (14.71 ± 3.78% [s.d.], z = 1.25, P unc = 0.21). States of sadness (13.49 ± 3.76% [s.d.], z = -3.31, P unc = 9.24E-4), fear (13.26 ± 3.42% [s.d.], z = -5.28, P unc = 1.28E-7), and anger (11.31 ± 3.62% [s.d.], z = -13.07, P unc = 4.78E-39) occurred with lower frequency, while states of contentment occurred the least often (8.74% ± 3.42% [s.d.], z = -19.61, P unc = 1.33E-85; see Table 1).

Although patterns of neural activation were most often classified as neutral as a whole, it is possible that consistent fluctuations in the time course of emotional states occur against this background. Research on MRI scanner-related anxiety has shown that self-report [22,23] and peripheral physiological [24] measures of anxiety peak at the beginning of scanning, when subjects first enter the scanner bore. This literature predicts that brain states indicative of fear should be most prevalent at the beginning of resting-state runs, and that neutral states should emerge over time, given their overall high prevalence (Fig 2B).

To assess gradual changes in the emotional states over time, we performed Friedman tests separately for each emotion category, all of which revealed significant effects of time (see S1 Table). Next, we quantified the direction of these effects using general linear models to predict classifier scores using scan time as an input. We found the scores for fear decreased over time ( , t 498 = -4.92, P unc = 1.20E-006, Fig 3 gray lines), whereas neutral states exhibited an increasing trend throughout the scanning period ( , t 498 = 7.36, P unc = 7.66E-013), consistent with predictions (additional effects were observed for scores for contentment [ , t 498 = 7.37, P unc = 7.05E-13], surprise [ , t 498 = 4.07, P unc = 5.51E-05], anger [ , t 498 = -3.36, P unc = 0.00085], and sadness [ , t 498 = -15.59, P unc < 2.52E-038]).

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Fig 3. Emotional states exhibit coherence during resting-state scans. Gray circles reflect the sample mean classification scores for all seven emotions (n = 499). Thick colored lines display group-average predicted time course using smoothing splines (with bordering 95% confidence interval). Text overlay (r cv ) indicates the average cross-validated correlation between splines fitted for each subject and tested on the average fit of other subjects. Dashed lines indicate linear fits over time. Solid black dots indicate time points at which a model has the highest proportion of classifications. Data are concatenated across two sessions of 256 s (solid vertical line). Note the early peak for fear scores and general increases in neutral scores over time. The data underlying this figure can be found in S1 Data. The raw fMRI resting state data can be obtained from https://www.haririlab.com/projects. https://doi.org/10.1371/journal.pbio.2000106.g003

To determine whether emotional states exhibited consistent dynamics over the course of the scanning period, we fit smoothing spline models [25] for each subject and assessed the correlation between each subject and the average time course of other subjects in a cross-validation procedure. This analysis showed that there is substantial moment-to-moment variability in the time course of emotional states across subjects (which cannot simply be explained by scaling differences in the emotion models or resting-state data; see S3 Fig). Consistent with the linear models using time as a predictor, evidence for neutral brain states was most prevalent in the second scanning session, especially during a peak at the beginning of the run, whereas the time course for fear peaked at the beginning of the first run and decreased throughout the scanning session. The model for surprise exhibited a similar time course as neutral states but peaked at the end of the second run. Additionally, this analysis showed that evidence for sad classifications peaked in the middle of the first run and decreased over time. Overall, these time series revealed a gradual change in evidence from negative emotions (fear and sadness in run 1) to non-valenced or bi-valenced emotions (neutral and surprise in run 2).

To ensure that our emotion-specific brain states are not proxies for more general resting-state networks thought to subserve other functions, we examined the spatial overlap between our models and those commonly derived by connectivity-based analysis of resting-state fMRI data [26]. On average, we observed little overlap (Jaccard index = 13.1 ± 1.97% [s.d.]; range 10.8%–16.7%) with the seven most prominent networks found in resting-state data, implicating a substantial degree of independence.

To further establish the construct validity of the spontaneous emotional brain states, we reasoned that their incidence should vary with individual differences in self-reported mood and personality traits associated with specific emotions. We assayed depressive mood with the Center for Epidemiologic Studies Depression Scale (CESD) [27] and state anxiety using the State-Trait Anxiety Inventory State Version (STAI-S) [28], instructing participants to indicate how they felt during the resting-state scan itself. Binomial regression models revealed that higher depression scores were associated with increases in the frequency of sadness ( , t 497 = 2.673, P unc = .0075, Fig 4A, see S4 Fig for scatter plots of predictions) and no other emotional state (all P unc > .24). State anxiety was associated with increasing classifications of fear ( , t 497 = 2.608, P unc = .0091) and decreasing frequency of contentment ( , t 497 = -2.015, P unc = .0439). Viewing these beta estimates as odds ratios (computed as ) reveals how a one-unit increase in self-reported mood is associated with differences in the occurrence of spontaneous emotional states. Applying this approach to CESD scores reveals that individuals with a score of 16 (the cutoff for identifying individuals at risk for depression) have 5.92% increased odds of being in a sad state compared to those with a score of 0. In more practical terms, this corresponds to approximately seven extra minutes a day of exhibiting a brain state that would be classified as sadness.

PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Fig 4. Individual differences in mood and personality modulate the occurrence of spontaneous emotional brain states. (A) Differences in depressive and anxious mood are associated with increases in the frequency of sad and fear classifications during rest. (B) Emotional traits of Anxiety, Angry Hostility, and Depression track differences in the frequency of fear, anger, and sad classifications (n = 499, error bars reflect standard error, * indicates effects significant at P unc < .05). The data underlying this figure can be found in S1 Data. The raw fMRI resting state data can be obtained from https://www.haririlab.com/projects. https://doi.org/10.1371/journal.pbio.2000106.g004