The primary aim of the present study was to investigate the association between chiropractic manipulative treatment and VBA stroke in a sample of the U.S. population. This study was modelled after a case–control design previously conducted for a Canadian population [32]. Administrative data for enrollees in a large national health care insurer were analyzed to explore the occurrence of VBA stroke across different time periods of exposure to chiropractic care in comparison with PCP care.

Unlike Cassidy et al. [32] and most other case–control studies [33,37,38], our results showed there was no significant association between VBA stroke and chiropractic visits. This was the case for both the commercial and MA populations. In contrast to two earlier case–control studies [32,33], this lack of association was found to be irrespective of age. Although, our results (Table 8) did lend credence to previous reports that VBA stroke occurs more frequently in patients under the age of 45 years. Additionally, the results from the present study did not identify a relevant temporal impact. There was no significant association, when the data were sufficient to calculate estimates, between chiropractic visits and stroke regardless of the hazard period (timing of most recent visit to a chiropractor and the occurrence of stroke).

There are several possible reasons for the variation in results with previous similar case–control studies. The younger (<45 yrs.) commercial cohort that received chiropractic care in our study had noticeably fewer cases. The 0–30 days hazard period included only 2 VBA stroke cases. There were no stroke cases for other hazard periods in this population. In contrast, earlier studies reported sufficient cases to calculate risk estimates for most hazard periods [32,33].

Another factor that potentially influenced the difference in results concerns the accuracy of hospital claims data in the U.S. vs. Ontario, Canada. The source population in the Province of Ontario was identified, in part, from the Discharge Abstract Database (DAD). The DAD includes hospital discharge and emergency visit diagnoses that have undergone a standardized assessment by a medical records coder [39]. To the best of our knowledge, similar quality management practices were not routinely applied to hospital claims data used in sourcing the population for our study.

An additional reason for the disparity in results may be due to differences in the proportions of chiropractic visits where SMT was reportedly performed. Our study showed that SMT was not reported by chiropractors in more than 30% of commercial cases. It is plausible that a number of the cases in earlier studies also did not include SMT as an intervention. Differences between studies in the proportion of cases reporting SMT may have affected the calculation of risk estimates.

Also, there were an insufficient number of cases having cervical and/or headache diagnoses in our study. Therefore, our sample population may have included proportionally less cases where cervical manipulation was performed.

Our results were consistent with previous findings [32,33] in showing a significant association between PCP visits and VBA stroke. The odds ratios for any PCP visit increase dramatically from 1–30 days to 1–1 day (Tables 6 and 7). This finding is consistent with the hypothesis that patients are more likely to see a PCP for symptoms related to vertebral artery dissection closer to the index date of their actual stroke. Since it is unlikely that the services provided by PCPs cause VBA strokes, the association between recent PCP visits and VBA stroke is more likely attributable to the background risk related to the natural history of the condition [32].

A secondary goal of our study was to assess the utility of employing chiropractic visits as a surrogate for SMT. Our findings indicate there is a high risk of bias associated with using this approach, which likely overestimated the strength of association. Less than 70% of stroke cases (commercial and MA) associated with chiropractic care included SMT. A somewhat higher proportion of chiropractic visits included SMT for the control groups (commercial = 76%; MA = 88%).

There are plausible reasons that support these findings. Internal analyses of claims data (not shown) consistently demonstrate that one visit is the most common number associated with a chiropractic episode of care. The single visit may consist of an evaluation without treatment such as SMT. Further; SMT may have been viewed as contraindicated due to signs and symptoms of vertebral artery dissection (VAD) and/or stroke. This might explain the greater proportion of SMT provided to control groups in both the commercial and MA populations.

Overall, our results increase confidence in the findings of a previous study [32], which concluded there was no excess risk of VBA stroke associated chiropractic care compared to primary care. Further, our results indicate there is no significant risk of VBA stroke associated with chiropractic care. Additionally, our findings highlight the potential flaws in using a surrogate variable (chiropractic visits) to estimate the risk of VBA stroke in association with a specific intervention (manipulation).

Our study had a number of strengths and limitations. Both case and control data were extracted from the same source population, which encompassed national health plan data for approximately 36 million commercial and 3 million MA members. A total of 1,829 cases were identified, making this the largest case–control study to investigate the association between chiropractic manipulation and VBA stroke. Due to the nationwide setting and large sample size, our study likely reduced the risk of bias related to geographic factors. However, there was a risk of selection bias – owing to the data set being from a single health insurer – including income status, workforce participation, and links to health care providers and hospitals.

Our study closely followed a methodological approach that had previously been described [32], thus allowing for more confident comparisons.

The current investigation analyzed data for a number of comorbid conditions that have been identified as potentially modifiable risk factors for a first ischemic stroke [40]. The differences between groups were statistically significant for most comorbidities. Information was not obtainable about behavioural comorbid factors e.g., smoking and body mass. With the exception of hypertensive disease, there are reasons to question the clinical significance of these conditions in the occurrence of ischemic stroke due to vertebral artery dissection. A large multinational case-referent study investigated the association between vascular risk factors (history of vascular disease, hypertension, smoking, hypercholesterolemia, diabetes mellitus, and obesity/overweight) for ischemic stroke and the occurrence of cervical artery dissection [41]. Only hypertension had a positive association (odds ratio 1.67; 95% confidence interval, 1.32 to 2.1; P <0.0001) with cervical artery dissection.

While the effect of other unmeasured confounders cannot be discounted, there is reason to suspect the absence of these data was not deleterious to the results. Cassidy, et al. found no significant differences in the results their case-crossover design, which affords better control of unknown confounding variables, and the findings of their case–control study [32].

Our results highlight just how unusual VBA stroke is in the MA cohort (prevalence = 0.021%) and – even more so – for the commercial population (prevalence = 0.0032%). As a result, some limitations of this study related to the rarity of reporting VBA stroke events. Despite the larger number of cases, data were insufficient to calculate estimates and confidence intervals for seven measures of exposure (4 commercial and 3 MA) for chiropractic visits. Additionally, we were not able to compute estimates specifically for headache and neck pain diagnoses due to small numbers. Confidence intervals associated with estimates tended to be wide making the results imprecise [42].

There were limitations related to the use of administrative claims data. “Disadvantages of using secondary data for research purposes include: variations in coding from hospital to hospital or from department to department, errors in coding and incomplete coding, for example in the presence of comorbidities. Random errors in coding and registration of discharge diagnoses may dilute and attenuate estimates of statistical association” [43]. The recordings of unvalidated hospital discharge diagnostic codes for stroke have been shown to be less precise when compared to chart review [44,45] and validated patient registries [43,46]. Cassidy, et al. [32] conducted a sensitivity analysis to determine the effect of diagnostic misclassification bias. Their conclusions did not change when the effects of misclassification were assumed to be similarly distributed between chiropractic and PCP cases.

A particular limitation in using administrative claims data is the paucity of contextual information surrounding the clinical encounters between chiropractors/PCPs and their patients. Historical elements describing the occurrence/absence of recent trauma or activities reported in case studies [47-51] as potential risk factors for VBA stroke were not available in claims data. Confidence was low concerning the ability of claims data to provide accurate and complete reporting of other health disorders, which have been described in case–control designs as being associated with the occurrence of VBA stroke e.g., migraine [52] or recent infection [53]. Symptoms and physical examination findings that would have permitted further stratification of cases were not reported in the claims data.

The reporting of clinical procedures using current procedural terminology (CPT) codes presented additional shortcomings concerning the accuracy and interpretation of administrative data. One inherent constraint was the lack of anatomic specificity associated with the use of standardized procedural codes in claims data. Chiropractic manipulative treatment codes (CPT 98940 – 98942) have been formatted to describe the number of spinal regions receiving manipulation. They do not identify the particular spinal regions manipulated.

Also, treatment information describing the type(s) of manipulation was not available. When SMT was reported, claims data could not discriminate among the range of techniques including thrust or rotational manipulation, various non-thrust interventions e.g., mechanical instruments, soft tissue mobilizations, muscle energy techniques, manual cervical traction, etc. Many of these techniques do not incorporate the same biomechanical stressors associated with the type of manipulation (high velocity low amplitude) that has been investigated as a putative risk factor for VBA stroke [54-56]. It seems plausible that the utility of future VBA stroke research would benefit from explicit descriptions of the particular type of manipulation performed.

Moreover, patient responses to care – including any adverse events suggestive of vertebral artery dissection or stroke-like symptoms – were not obtainable in the data set used for the current study.

In the absence of performing comprehensive clinical chart audits, it is not possible to know from claims data what actually transpired in the clinical encounter. Further, chart notes may themselves be incomplete or otherwise fail to precisely describe the nature of interventions [57]. Therefore, manipulation codes represent surrogate measures, albeit more direct surrogate measures, than simply using the exposure to chiropractic visits.

Our study was also limited to replication of the case–control design described by Cassidy, et al. [32]. For pragmatic reasons, we did not attempt to conduct a case-crossover design. While the addition of a case-crossover design would have provided better control of confounding variables, Cassidy, et al. [32] showed the results were similar for both the case control and case crossover studies.

The findings of this case–control study and previous retrospective research underscore the need to rethink how to better conduct future investigations. Researchers should seek to avoid the use of surrogate measures or use the least indirect measures available. Instead, the focus should be on capturing data about the types of services and not the type of health care provider.

In alignment with this approach, it is also important for investigators to access contextual data (e.g., from electronic health records), which can be enabled by qualitative data analysis computer programs [58]. The acquisition of the elements of clinical encounters – including history, diagnosis, intervention, and adverse events – can provide the infrastructure for more actionable research. Because of the rarity of VBA stroke, large data sets (e.g., registries) containing these elements will be necessary to achieve adequate statistical power for making confident conclusions.

Until research efforts produce more definitive results, health care policy and clinical practice judgments are best informed by the evidence about the effectiveness of manipulation, plausible treatment options (including non-thrust manual techniques) and individual patient values [20].