Data on police killings were obtained from the Mapping Police Violence (MPV) database, which has tracked police killings in the USA since 2013.The MPV database draws from several media and crowd-sourced databases for police killings, which helps to address concerns about under-reporting in government statistics.Suspected killings are cross-checked against police and criminal records, social media, and obituaries. The database includes information on age, race, and sex of the victim; whether the victim was armed (with any weapon that could be used to harm or kill others); and location of the killing. We used data from the MPV database up to and including 2016 to better quantify the numbers of police killings occurring both before and after the BRFSS interview dates.

We obtained data on respondents from the US Behavioral Risk Factor Surveillance System (BRFSS), a nationally representative, telephone-based, random digit dial survey of non-institutionalised adults aged 18 years and older. We used self-reported race to identify black American respondents. We also extracted information on respondent sex, age, and educational attainment. We used the 2013–15 BRFSS to match available data on the timing of police killings. We note that some respondents in the 2015 BRFSS sample were in fact interviewed in early 2016.

We also constructed a binary exposure variable equal to one if there had been one or more police killings of unarmed black Americans in the same state during the 3 months prior to the BRFSS interview (vs none). Finally, to more precisely elucidate the timing of the effects on mental health, we defined a series of exposure variables representing the number of police killings of unarmed black Americans in the same state in each of the 6 months preceding and after the BRFSS interview date.

The primary exposure was the number of police killings of unarmed black Americans occurring in the 3 months prior to the exact date of the BRFSS interview in the same state. BRFSS respondents were defined as exposed on the basis of their state of residence and the interview date. Thus, this exposure definition combines a range of potential exposure mechanisms including word of mouth and stories in print, radio, television, and social media. State of residence was used to define the exposure for several reasons. First, it is the smallest geographical area identified in the public-use BRFSS data files. Second, Google search frequencies of victims' names from recent high-profile police killings of unarmed black Americans suggest that these events have their greatest salience in the state media markets in which they occur ( appendix p 4 ). Third, existing literature shows a strong relationship between state-level measures of racism and police killings of unarmed black Americans.Fourth, police killings might be seen to reflect local police–community relationships and state legal environments. Behavioural responses to police violence correlate strongly with locally reported cases.

The primary outcome was the number of days in the previous month in which the respondent's mental health was reported as “not good”. The specific question from the BFRSS used to measure respondents' mental health was worded as follows: “Now thinking about your mental health, which includes stress, depression, and problems with emotions, for how many days during the past 30 days was your mental health not good?” This survey item has been shown to have a high degree of internal validity, construct validity, criterion-related validity, and test-retest reliability, and is widely used to monitor trends in population mental morbidity.We also considered two secondary outcomes: whether or not an individual reported at least 1 day of poor mental health (vs none) and whether or not an individual reported 14 or more days of poor mental health (vs less than 14 days) in the prior month. The latter measure has been widely used to quantify the population prevalence of frequent mental distress because it correlates with severe mental health conditions.

Statistical analysis

28 Zou G A modified Poisson regression approach to prospective studies with binary data. To estimate the effect of police killings of unarmed black Americans on self-reported mental health in the general population of black American adults, we fit difference-in-differences multivariable regression models specifying the number of poor mental health days as the outcome variable and the number of police killings of unarmed black Americans in the 3 months prior to interview as the primary exposure of interest. We estimated least-squares regression models for the primary outcome and report regression coefficients, which are interpretable as rate differences. We also estimated robust Poisson models for the primary and secondary outcomes, with the exponentiated regression coefficients interpretable as relative rates and risks,respectively. In all models, we adjusted for year-month fixed effects to account for national secular trends in the outcome; state-month fixed effects to account for time-invariant state-level confounders and state-specific seasonality in mental health; day-of-week fixed effects; and individual-level age group (ranging from 18–24 years to 80 years and older in 5-year intervals), sex, and level of education fixed effects. We also estimated models replacing the 3-month exposure variable with the binary measure of any police killing of an unarmed black American in the past 3 months. Further details on our methods are provided in the appendix (pp 2–3)

29 Angrist JD

Pischke JS Our models compare the mental health of black Americans surveyed after a police killing of an unarmed black American in the same state with the mental health of black Americans residing in the same state but surveyed before that event or more than 3 months after the event, adjusting for state-specific seasonal patterns in mental health and for temporal trends in mental health of black Americans living in other states. The causal identifying assumptions in our strategy are (1) no endogenous selection into the sample—ie, the timing of the BRFSS interview and participation in the interview were random relative to the timing of police killings, an assumption supported by the random-digit-dial approach to sampling and by our investigation of differential participation following police killings; and (2) unconfoundedness—ie, police killings did not coincide with other factors that could influence mental health. After including state-month and year-month fixed effects, the only unobserved confounders that could remain would be factors that varied at the state-year-month level and whose timing was correlated with police killings and mental health in ways that deviated from normal state-specific seasonal patterns. The unconfoundedness assumption is supported by the quasi-random timing of specific police killings. To the extent that these assumptions are satisfied—which we explored in several sensitivity analyses—our estimates can be interpreted as causal effects.

We note that our analysis defines as exposed any person living in the state where the police killing occurred, including some people who might not have been substantively exposed to the police killing (or even aware of it). Our causal estimates therefore have an intention-to-treat interpretation, similar to a randomised trial with non-compliance to treatment assignment. Our models capture the aggregate population-level effect, but probably underestimate the effect of police killings on the mental health of subpopulations most directly exposed.

30 Cameron AC

Miller DL A practioner's guide to cluster-robust inference. All models were estimated using standard BRFSS survey weights to account for survey design and differential non-response. The overall response rate of the BRFSS ranges between 40% and 45% each year, which is comparable to, if not higher than, other large population-representative telephone surveys in the USA. We computed heteroscedasticity-robust SEs corrected for clustering at the state level. Clustered SEs adjust for spatial and temporal error correlation among respondents within the same state. We also calculated alternate SEs and p values using the wild cluster bootstrap-t method, which addresses the potential for over-rejection of the null hypothesis when the number of cluster units is small.

We conducted four sensitivity analyses to assess the assumptions needed to infer causality in our model. First, to investigate the possibility of residual confounding, we estimated models replacing the 3-month exposure variable with 12 count variables denoting exposures in each of the 6 months before and after interview. This model allows a prespecified falsification test: if there were unobserved confounders at the state-year-month level, then we would expect the changes in mental health to sometimes precede the actual killing of an unarmed black American, given that there is randomness in the specific timing of these events. Second, we estimated models that additionally included state-year fixed effects or census division-month-year fixed effects (census divisions represent nine geographical regions accounting for the 50 US states). These additional models adjusted for state-specific year-on-year secular trends (eg, worsening crime) and regional shocks (eg, time-limited weather events), respectively, ruling out additional classes of potential confounders. Third, to assess the potential for bias due to non-random participation in BRFSS as a function of exposure to police killings, we re-fit the regression models specifying individual-level demographic characteristics (respondent sex, age, and educational attainment) as the dependent variables. Fourth, we estimated models that additionally included a measure of participant income (which was reported by only 85% of participants).

To assess the specificity of our findings to black Americans, we estimated the effects of police killings of unarmed black Americans on the mental health of white respondents. We also estimated the effects of police killings of armed black Americans and unarmed white Americans on the mental health of both black and white respondents. These analyses help to elucidate the mechanisms accounting for our findings. If police killings of unarmed black Americans affected the mental health of other black Americans through heightened perceptions of structural racism, activation of prior traumas, or through racial identification with the deceased, then we would expect no effect on the mental health of white Americans, nor in any of the cross-race analyses. By contrast, if police killings of unarmed white or black Americans were broadly interpreted as signals of general threats to personal safety, we would expect meaningful cross-race effects of these exposures. Although police killings of armed victims could affect mental health via communal bereavement, these killings are less likely to be perceived as unjustified and we expected little effect on mental health. We did not ex ante specify models examining the consequences of police killings of armed white Americans because our intent was to assess the sensitivity of our findings to changes in the race or armed status of the victim, holding fixed at least one of these attributes. These specifications provide the sharpest, most stringent test of whether killings of black, unarmed victims have specific salience to health outcomes.

Finally, we conducted two additional analyses to guide interpretation of the results. First, we estimated models differentiating between exposure to one police killing versus exposure to two or more. These models help to distinguish between threshold-related versus dose-related consequences of these events. Second, we used subgroup analyses to investigate whether the estimated mental health impacts were confined to a particular subset of black Americans. We stratified the estimates by sex (male or female), age (18–34 years, 35–49 years, 50–64 years, or 65 years and older), education (less than high school, high school graduate, or at least some college), and household income (above and below the sample median: <US$35 000 or ≥$35 000), and tested the null hypotheses that the estimated effects were the same in each subgroup. Unlike our primary specifications, these analyses were not based on a-priori hypotheses and therefore should be regarded as hypothesis-generating exercises. Statistical power to detect meaningful differences across subgroups was limited by sample size.