Inclusion of herd-protection effects in CEAs had a substantial impact in the estimated ICERs and made target-vaccination strategies more attractive options in almost half of the cases where ICERs were above the societal-willingness to pay threshold without herd-protection. More empirical epidemiologic data are needed to determine the size of herd-protection effects across diverse settings and also the size of negative vaccine effects, e.g. from serotype substitution.

We identified 35 CEA studies (20 pneumococcal, 4 meningococcal, 8 rotavirus and 3 influenza vaccines) with 99 ICER-analyses (55 per-QALY, 27 per-LY and 17 per-DALY). The median ICER-absolute differences per QALY, LY and DALY (without minus with herd-protection) were $15,620 (IQR: $877 to $48,376); $54,871 (IQR: $787 to $115,026) and $49 (IQR: $15 to $1,636) respectively. When the target-vaccination strategy was not cost-saving without herd-protection, inclusion of herd-protection always resulted in more favorable results. In CEAs that had ICERs above the cost-effectiveness threshold without herd-protection, inclusion of herd-protection led to crossing of that threshold in 45% of the cases. This impacted only CEAs for more developed countries, as all but one CEAs for less developed countries had ICERs below the WHO-cost-effectiveness threshold even without herd-protection. In several analyses, recommendation for the adoption of the target vaccination strategy depended on the inclusion of the herd protection effect.

We performed a quantitative comparative analysis of the impact of herd-protection effects in CEAs for four childhood vaccinations (pneumococcal, meningococcal, rotavirus and influenza). We considered CEAs reporting incremental-cost-effectiveness-ratios (ICERs) (per quality-adjusted-life-years [QALY] gained; per life-years [LY] gained or per disability-adjusted-life-years [DALY] avoided), both with and without herd protection, while keeping all other model parameters stable. We calculated the size of the ICER-differences without vs with-herd-protection and estimated how often inclusion of herd-protection led to crossing of the cost-effectiveness threshold (of an assumed societal-willingness-to-pay) of $50,000 for more-developed countries or X3GDP/capita (WHO-threshold) for less-developed countries.

Vaccine herd-protection has been reported for several childhood vaccinations including pneumococcal (e.g. PCV7, PCV10 and PCV13) [ 13 – 18 ] meningococcal, [ 11 , 19 – 21 ] rotavirus [ 22 , 23 ] and influenza vaccines. [ 24 , 25 ] We evaluated the overall impact of including herd-protection assumptions in CEAs for these four childhood vaccinations. We addressed the following questions: How often does the inclusion of herd-protection change the conclusions of CEAs and give favorable results for the target vaccination strategy? How large is the impact of including herd-protection in CEAs? How often does the inclusion of herd-protection drive estimates below the willingness-to pay cost-effectiveness-thresholds? Is there a consistent pattern of herd-protection impact across these four vaccines? And finally, is the impact of herd-protection larger in CEAs for more-developed countries and when industry is involved?

Cost effectiveness analysis (CEA) studies [ 1 ] have been increasingly used worldwide and in the US in particular [ 2 , 3 ] for the development of national immunization strategies. CEA conclusions can be affected by different methodological choices, modeling choices and populations targeted. Baseline vaccine efficacy assumptions can be influential [ 4 – 12 ]. Of particular interest is the potential impact of indirect vaccine effect assumptions, and specifically vaccine herd-protection (positive effect) and serotype substitution (negative effects) in the community. Herd-protection is the reduction of the disease in non-vaccinated susceptible individuals from widespread humoral immunity and/or decreased carriage (e.g. nasopharyngeal carriage) in vaccinated individuals in the community, that lead to decreased likelihood of non-vaccinated individuals having contact with infected/infectious individuals [ 13 ]. Although this phenomenon is widely described, empirical epidemiologic data on the size of indirect vaccine effects are limited. Vaccine CEAs that include indirect vaccine effects in their analyses either use modeling or extrapolate data from studies conducted in other countries, which may have different disease epidemiology.

In cases where the comparator-strategy was no vaccination we also calculated how often the inclusion of herd-protection led to crossing the cost-effectiveness threshold, from an ICER above that threshold without herd-protection to an ICER below that threshold with herd-protection [ 34 ]. For more-developed countries a threshold of $50,000 (or ~£30,000 respectively), representing the assumed societal willingness-to-pay for a QALY (or LY) gained, is often used [ 35 – 38 ]; while for less-developed countries the WHO-cost-effectiveness-threshold of X3GDP/capita (gross domestic product per capita) is often used (Text B in S1 File ) [ 39 ]. In exploratory analyses, we also evaluated whether the number of ICER-analyses crossing the cost-effectiveness threshold (without vs with herd-protection) differed according to country setting (more-developed versus less-developed countries, as defined in Figure A in S1 File ); industry involvement; perspective (societal versus health care) and model (dynamic vs static).

For each compared vaccination strategies we calculated the absolute differences of ICERs per-QALYs gained, of ICERs per-LY gained and of ICERs per-DALYs avoided, “without herd-protection” minus “with herd-protection.” A difference with a positive value indicates that the ICER with herd-protection was more favorable than the ICER without herd-protection. We evaluated the pattern of impact of herd-protection in CEAs across different ICER-metrics and across difference vaccines (median and inter-quartile ranges of ICERs’ differences) and compared ICER-differences across metrics and across vaccines by the non-parametric Kruskal-Wallis test.

For the analyses “with herd-protection” (if multiple scenarios were reported), we always selected the scenario closest, in terms of additional parameter assumptions, to the base-case scenario without herd-protection. In 3 pneumococcal-conjugate vaccine [ 30 – 32 ] and 1 rotavirus vaccine study [ 33 ] where different herd-protection assumptions were considered, we decided a priori to keep the analyses for a herd-protection rate closest to 15%. Moreover, we considered only analyses that included herd-protection for both of the compared vaccination strategies.

For our “without herd-protection” analyses we used the base-case scenario if it was clearly defined as without herd-protection. If the base-case scenario was not clearly defined, we selected the scenario that was without herd-protection and had the least number of additional assumptions for other parameters (e.g. for discount rates, vaccination coverage rates, waning vaccination immunity, etc.). We considered studies using either static models (provided that sensitivity analyses with herd-protection were also considered in addition to analyses without herd-protection) or dynamic models. If the study used a dynamic model and the base-case scenario already included herd-protection, we selected sensitivity/subgroup/sub-model/or scenario analyses that were “without herd-protection” and had the same assumptions for all other parameters as the base-case scenario.

From each eligible study we extracted the following information: first author; journal; year; compared interventions (target vaccination strategy versus comparator vaccination strategy or no vaccination [for the characterization of vaccination strategies as target or comparator we used the authors’ definitions and if this was unclear, we considered as target the most recently approved vaccine]); perspective for the cost-analysis (societal or health care system); model (static vs dynamic model [dynamic models are able to produce empiric results influenced by herd-protection indirect-vaccine-effects while static models rely on assumptions for herd-protection]); target population (cohort model vs population model); vaccination coverage rates; monetary unit used (currency and year); industry involvement; size of assumed/modeled herd-protection effect (and reference(s) cited thereof); quantitative ICERs with and without herd-protection per-QALYs gained, per-LYs gained, and per-DALYs avoided (between the compared vaccination strategies; [negative ICERs indicated cost-saving with the target vaccination strategy vs. the comparator]) and the authors’ interpretation of the CEAs findings (the target vaccination strategy was recommended, not recommended, or statement was unclear). Pertinent quantitative data presented only in figures were also extracted using the WebPlot Digitizer software [ 27 ]. Estimates in foreign currencies were converted to US dollars for the same year using the OANDA’s currency calculator tool [ 28 ]. All values were subsequently inflated to 2016 US dollars to allow comparability of results [ 29 ].

We searched PubMed and the Tufts CEA registry [ 26 ] (last search was January 1, 2014; search-strategy in Text A in S1 File ). For the Tufts CEA Registry searches we entered the type of vaccine (i.e. influenza vaccine) into the basic search function. We also screened the reference list of prior systematic reviews of CEA for the four targeted childhood vaccinations. Reviews, commentaries, editorials, letters, abstracts from meetings and articles published in non-English language were excluded. Articles were screened at Title/Abstract level and potentially eligible articles were reviewed in full text.

In this evaluation we considered economic analyses characterized by their authors as CEAs, cost-utility analyses or cost-benefit analyses. We will use the term CEA for consistency unless stated otherwise. We included analyses published in English that targeted childhood vaccination strategies for pneumococcal, meningococcal, rotavirus or influenza disease in infants, children or adolescents. Analyses of strategies vaccinating only adults were excluded. We further considered those studies that reported incremental cost effectiveness ratios (ICER) between the target vaccination strategy and the comparator vaccination strategy (or no vaccination) both with and without herd-protection. CEAs that included herd-protection in their base-case scenario (or primary model) were considered eligible if they also performed separate sensitivity/subgroup/sub-model/scenario analyses without herd-protection. Analyses that reported only cost-effectiveness ratios for single vaccination strategies, and not incremental cost-effectiveness ratios between compared vaccination strategies, were excluded. Data for composite indirect vaccine effects including both herd-protection (positive effect) and serotype substitution (negative effect) or only serotype substitution were not included in our primary analyses (data were very limited to allow for meaningful separate analyses).

The final authors’ conclusions appear in Table F in S1 File . The authors clearly recommended at least one target vaccination strategy in 69% (24/35) of studies and in another six studies considered that they could have been cost-effective under certain assumptions (herd-protection was considered among the key assumptions that would have changed the conclusions in 4 studies) (Table D in S1 File ). Finally, 5 studies did not recommend the target vaccination strategies. Furthermore, the target vaccination strategy was clearly recommended in 83% (20/24) of industry-funded studies vs. 55% (6/11) of non-industry-funded studies (p = 0.07).

The differences in ICERs per-QALYs (without vs with herd-protection) across all four vaccines are depicted in the box-plot in Figure B-2 in S1 File and at the bar-plots in Fig 2 and Figure C in S1 File . (p = 0.0002 by Kruskal-Wallis). The respective differences for ICERs per-LYs and per DALYs across vaccines were not statistically significant (p = 0.79 and p = 0.37 respectively by Kruskal-Wallis) (Figure D-1 in S1 File and Figure E-1 in S1 File ).

The scatterplot of the three ICER-metrics, with vs without herd-protection, is shown in the in Fig 3 . The ICER-differences varied significantly according to the metric used (p = 0.0006 by Kruskal-Wallis, for ICER-differences per-QALYs vs per-LYs vs per-DALYs) (Figure B-2 in S1 File ). Moreover, in 59% (10/17) of ICER per-DALYs analyses the differences were <$100 [ 33 , 48 ] ( Table 2 ). These studies were in low-income countries and according to the World Bank, for low-income countries (with GDP/capita ≤ $735), interventions with ICERs per-DALY ≤$150 are considered attractive [ 71 ] ( Table 2 ).

There was no difference in the number of ICER-outcome-analyses that crossed the cost-effectiveness threshold of $50,000 for more-developed countries and X3GDP/capita for less-developed countries (without vs with herd-protection) according to industry involvement, perspective analyzed or model used (among those that were above that threshold without herd-protection and were comparing a target vaccine against no vaccine). These thresholds were crossed in 50% (8/16) of those analyses with industry-involvement vs 41% (9/22) without industry-involvement (p = 0.58); in 44% (7/16) of those analyses with the societal perspective vs 45% (10/22) with the healthcare-perspective vs (p = 0.92) and in 54% (13/24) of those analyses with static-models vs 29% (4/14) with dynamic models (p = 0.13) (Table D in S1 File ).

This phenomenon was observed only in analyses for more-developed countries. In all but one of the 29 ICER-analyses for less-developed countries, the target vaccination strategy was already below the cost-effectiveness threshold of X3GDP/capita even without inclusion of herd-protection. The ICER per-LY analysis for pneumococcal vaccine PCV7 for Brazil [ 55 ] was above the X3GDP/capita cost-effectiveness threshold and remained slightly above that threshold with herd-protection (the ICER per-DALY analysis for the same study was below the X3GDP/capita threshold without herd-protection).

Overall, among 79 ICER-analyses that compared target vaccine strategies vs no vaccination, 48% (38/79) had ICERs that without herd-protection were above the cost-effectiveness threshold of $50,000 for more developed countries and X3GDP/capita for less developed countries (37 ICER-analyses for more-developed countries and 1 ICER-analysis for a less-developed country) (Table D in S1 File ). (The WHO-thresholds of X3GDP/capita reported in individual studies for less developed countries are shown in Text B in S1 File ).

When the target vaccination strategies were not already cost saving without herd-protection (n = 83/99), ICERs were always more favorable with inclusion of herd-protection ( Table 2 and Table D in S1 File ). Among the 16 ICER-analyses that were already cost saving without herd-protection (12 ICER per-QALYs and 4 ICER per-LYs analyses), inclusion of herd-protection gave additionally more favorable cost-saving results in 31% (5/16) of those.

Industry was involved in 69% (24/35) of the CEA studies (and in 64% [63/99] of ICER-outcome-analyses respectively) ( Table 1 and Table A in S1 File ); the healthcare perspective was analyzed in 70% (69/99) of ICER-outcome-analyses and static models were used in 83% (29/35) of the CEA studies (and 73% [72/99] of ICER-outcome analyses respectively). Additional characteristics of the included studies and ICER-analyses, including the models, assumed vaccination coverage rates and herd-protection assumptions, are described in detail in Table 1 and Tables A- C in S1 File .

We screened 469 articles and identified 35 eligible studies [ 30 – 33 ; 40 – 70 ] (20 for pneumococcal conjugate vaccines; 4 for meningococcal vaccines (3 conjugate vaccines and 1 polysaccharide vaccines); 8 for rotavirus and 3 for influenza-virus vaccines) with a total of 81 separate analyses for different compared vaccination strategies, countries, and/or perspectives (37 pneumococcal, 13 meningococcal, 22 rotavirus and 9 influenza vaccination strategies analyses); and a total of 99 ICER-analyses (n = 55 per-QALYs [26 studies]; n = 27 per-LYs [15 studies] and 17 per-DALYs [5 studies] ( Fig 1 , Table 1 and Table A in S1 File ). Each study could have reported more than one of these ICERs.

Discussion

In this quantitative comparative analysis of the incremental-cost-effectiveness-ratios for four childhood vaccinations with vs without vaccine herd-protection, we showed that inclusion of herd-protection effects had a substantial impact in the estimated ICERs. In cases where the ICERs were above the cost-effectiveness threshold (of assumed societal-willingness-to-pay of $50,000 for more-developed countries or X3GDP/capita for less-developed countries) without herd-protection, inclusion of herd-protection led to crossing of that threshold in 45% of cases, making the target vaccination-strategy more attractive option. This impacted only CEAs for more-developed countries, as all but one of CEAs for less-developed countries had ICERs below the WHO-cost-effectiveness threshold even without herd-protection. We were not able to draw robust conclusions for true differences among those crossing the above thresholds with herd-protection, according to vaccine type, industry involvement status, perspective of analysis and CEA-model used due to the small numbers of analyses within individual subgroups. Among analyses that were not already cost saving without herd-protection, inclusion of herd-protection always gave more favorable results. Moreover, in a third of analyses that were already cost saving, the inclusion of herd-protection gave additionally more favorable cost-saving results.

The ICER-differences varied significantly according to the metric used, with the largest differences seen with the ICERs per-QALYs and per-LYs. The ICER-differences per-DALYs were small; however, even without herd-protection, the ICERs per-DALYs estimates were significantly smaller than ICERs per-QALYs or ICERs per-LYs estimates. In several studies, recommendation for the adoption of the target vaccination strategy depended on the inclusion of herd-protection effects in the calculations. Moreover, the ICER per-QALY differences varied according to vaccine, with the largest differences seen with pneumococcal and meningococcal vaccines. A possible explanation for that could be that the herd-protection effects considered for the pneumococcal and meningococcal vaccines were overall larger than those for the rotavirus and influenza vaccines; however the herd-protection assumptions considered varied significantly across studies even within the same vaccine group as shown in Table B in S1 File. Although no significant differences were detected across vaccines in the ICER-differences per LYs and per-DALYs, the data were more limited.

Empirical epidemiologic data on the size of vaccine-herd-protection effects in different countries were limited and most studies extrapolated herd-protection assumptions from other countries, used fixed herd-protection assumptions or applied modeling [72]. A prior systematic review by van de Vooren et al. [73] showed that among 10 European pneumococcal-conjugate vaccine CEA studies, only one study based herd-protection assumptions on national data, while most of the remaining studies used information for herd-protection and serotype substitution based on an American study. Although the approach of using assumptions rather than actual epidemiologic data for herd-protection might be appropriate for economic evaluations in settings where a vaccine is still being considered (2/10 studies), the majority of these studies were done in countries where the vaccines were already recommended [73]. Extrapolation of herd-protection effects from different countries should be cautiously done as differences in the dominant circulating strains, transmissibility of strains and other social factors (e.g. social mixing situations) as well as differences in the vaccination dosing schedules and vaccination coverage rates may impact the herd-protection effects [19]. Loo et al. [17] showed that indirect vaccine effects from pneumococcal-conjugate vaccines vary widely according to dosing schedule and endpoint studied (e.g. vaccine-serotype associated invasive pneumococcal disease, nasopharyngeal pneumococcal carriage, and pneumonia). This suggests that in order to be most useful, future CEA methodology must account for this complicated epidemiology.

It has been previously shown that most published economic-analyses (not limited to vaccines) reported favorable ICERs for the experimental interventions [74] and industry-sponsored economic-analyses in particular were more likely to report favorable ratios compared to non-industry sponsored CEA studies [74–76] In our sample of evaluated vaccine-CEAs the non-industry funded CEAs studies were less than a third of the total number of studies, to allow for detection of true between-group differences.

Our quantitative comparative analysis differs from prior systematic reviews of CEA studies for childhood vaccinations, as these prior reviews were mainly qualitative descriptive reviews. A detailed discussion of the differences between our analysis and these prior reviews is included in Text C in S1 file. In brief, in our quantitative comparative analysis with vs without herd protection: a) we applied very strict criteria for the ICER-analyses to be compared to optimize comparability of results (considering only ICER-analyses where all other parameters/assumptions, except for herd-protection, were the same); b) we calculated the size of the ICER-differences with vs without herd-protection across diverse childhood vaccines and for different ICER-metrics (per-QALYs, per-LYs and per-DALYs) to increase our power to detect true differences; c) we inflated all monetary-values to 2016 US dollars to increase comparability of results across studies and d) we explored factors that could explain the observed differences in the impact of herd-protection across studies, such as country setting, industry involvement status, CEA-perspective, vaccine type and CEA-model used.

Some study limitations should be acknowledged. We analyzed only the positive indirect vaccine effects from herd-protection. However, available data were very limited to allow for the performance of meaningful separate analyses for the impact of negative indirect vaccine effects, such as serotype substitution (either alone or in combination with herd-protection), in the ICER estimates. Only five pneumococcal vaccine CEA-studies included data with and without additional vaccine indirect effects. (We discussed those in detail in Text D in S1 File). We only analyzed economic analyses published in English; pertinent studies published in other languages [77, 78] might have been missed. We used benchmark cost-effectiveness-thresholds to assess the impact of including herd-protection in vaccine CEAs for more-developed countries. These thresholds are arbitrary [34, 79] but nevertheless are widely used to characterize interventions as cost-effective and worth adopting. Moreover, for less-developed low-income countries we applied the widely used WHO cost-effectiveness-thresholds of X3GDP/capita [39].

Overall, there is a need for continued surveillance and collection of robust empirical epidemiologic data on herd-protection positive vaccine effects and negative indirect vaccine effects e.g. from serotype substitution across diverse populations, countries, for different vaccination dosing schedules and vaccination coverage rates. Moreover, further methodological research is needed for the identification of the most efficient methods for incorporating herd-protection effects in economic analyses [19] There is progress towards that direction as international guidelines for the standardization of economic evaluations for vaccines have been recently developed by the European Vaccine Economics Community [80]. Dynamic models should be preferably used in those analyses, instead of static models, as they include the interaction between individuals and therefore account for indirect vaccine-effects [80]. The routine inclusion in vaccine economic analyses of the negative vaccine indirect effects, such as serotype substitution, in addition to the positive herd-protection effects, is necessary. Especially in the case of pneumococcal vaccination there is a need for more sophisticated models that count for carriage of different serotypes and not only for infection or illness. New approaches for the assessment of vaccine herd-protection, such as cluster-randomized trials that can assess vaccine-direct effects, herd-protection effects and negative indirect effects, e.g. from serotype replacement, even before the introduction of vaccines into public health programs, should be considered [81]. Moreover, pre-licensure assessment of vaccine herd-protection should not be used as a replacement for post-licensure assessments, as only post-licensure studies in diverse populations can provide an accurate estimate of vaccine’s herd-protection effects [81–84].