Overall conceptual approach

Four key sources of data were needed to estimate the IRR of the NMB of the health gains arising from cancer research:

a time series of the public and charitable funding of cancer-related research;

a time series of the NMB of cancer health gains, derived from the monetised health benefits and the healthcare costs for selected interventions a ;

an estimate of the elapsed time between the investment (research funding) and return (health gain) associated with those interventions; and

an estimate of the amount of health gain that should be attributed to public and charitable research investment in cancer-related research in the UK.

With these four data inputs, we then calculated a rate of return on the investment in cancer research.

It should be noted that the costs of private sector R&D investments are accounted for in our analysis as elements within the cost of delivering health care, which are netted off in the NMB. The costs to the health service of medical interventions produced by the private sector include the return to the private sector on its R&D investments.

Estimating public and charitable funding of cancer-related research

The leading funders of cancer research in the UK were identified by examining the National Cancer Research Institute (NCRI) Cancer Research Database. Between 2002 and 2011, the top 10 funders consistently accounted for over 95% of cancer research spend by the 21 NCRI partners.b Estimates of annual cancer-related research funding between 1970 and 2009 were assembled for these 10 organisations plus an estimated contribution to cover Funding Council support for cancer research (the Higher Education Funding Council for England and similar bodies in Wales, Scotland and Northern Ireland provide a performance-related block grant to UK universities based on the quality and volume of research). A detailed account of how we estimated these 11 time series is provided (see Additional file 1).

As also discussed in detail in Additional file 1, in estimating research spend for the Funding Councils and the Department of Health (DH)/NHS, we had to derive a figure specifically for cancer-related research activity in the UK. We settled on a central estimate of 10% of total publicly and charitably funded health and biomedical research activity, and we also assumed it to be constant over the time period. This estimate was derived from a number of independent sources, as follows

Medical Research Council (MRC) spending on cancer research averaged 9.8% of their total investment (range: 4.6% to 16.7%) between 1970/1 and 2009/10.

Wellcome Trust cancer funding was more erratic, ranging between 1% c and 38%, with an average of 14.5% of expenditure being on cancer research.

The proportion of peer-reviewed research papers in oncology as a percentage of all UK biomedical outputs averaged 9.2% (range: 8.5% to 9.5%) between 1988 and 1995 [25].

The proportion of peer-reviewed research papers in oncology research (as a percentage of all NHS research outputs) was 12% between 1990 and 1997 [26].

The proportion of mainstream quality-related (QR) funding allocations by the Higher Education Funding Council for England for ‘Cancer studies’ (that is, Unit of Assessment 02) between 2009 and 2012 was around 6% of the total biomedical allocation (that is, Unit of Assessments 01 to 15 and 44).d

Given the importance of this estimate of 10% for the proportion of research activity that is related to cancer (for those sources where we had no actual breakdown), we also looked at the effect of lower and higher estimates of 7.5% and 15%, respectively, in the sensitivity analyses.

Estimating the NMB from cancer-related research

This element of the research required estimates of the lifetime QALYs gained and the net lifetime costs to the NHS of delivering those QALYs for research-based interventions provided in each year of the period 1991 to 2010. The general methods mirrored those used in the 2008 study [3] on the returns on investment in CVD research, and again built up the aggregate net benefits from the bottom up, aggregating the QALYs gained and the net NHS costs from the use of specific interventions. This approach required: 1) identification of the key relevant cancer interventions and their level of usage during the relevant period; and 2) estimates of the QALY gains and NHS costs associated with the interventions. From this information, the NMB was calculated as the health benefit valued in monetary terms (determined by the quantity of health benefit and a decision-maker’s willingness to pay for that additional benefit) minus the cost of delivering that health benefit.

In the CVD study, our starting point was previously published research identifying the cardiovascular interventions that had contributed most health gain [27]. No equivalent studies for cancer were identified that could provide a comparable basis for deciding which interventions were, quantitatively, the most important to include in the analysis. Thus, the three main steps for quantifying the total NMB associated with cancer interventions were: 1) to identify the cancer interventions that were the likely major sources of benefits; 2) to identify appropriate estimates of NMB per patient for that subset of cancer interventions; and 3) to construct a time series (for 1991 to 2010) of the number of patients receiving each of these subsets of cancer intervention in the UK.

Identifying the key cancer interventions

At the outset of the study, we had a number of discussions with cancer research experts to provide us with a broad understanding of the main developments in the field over the past 20 years. Informed by these discussions, we quantitatively identified those areas that had resulted in the largest health gain in the UK since 1990, arising from three main sources: 1) key cancers where research and resultant health policies have led to health gains through a reduction in incidence; 2) key cancers for which screening programmes have led to health gains because of early detection; and 3) key cancers where there have been the most significant health gains from increased survival.

To identify areas where a reduction in incidence has been observed, cancer incidence data in the UK were analysed, using UK incidence rates between 1990 and 2008 [28], to calculate a percentage change over the period. This percentage change was then multiplied by mid-period UK incidence (the average per year for 1999 to 2001 [29]) to estimate an absolute change in incidence. Four cancer types have seen significantly larger reductions in incidence between 1990 and 2008: lung (6,500), stomach (4,400), bladder (4,400), and cervical (1,400) cancers. Additional file 2 gives full details for the 21 cancers. The literature was consulted to identify possible causes for these reductions in incidence. Overwhelmingly, smoking prevention and cessation was cited as the reason for a reduction in lung cancers [30]. Falls in rates of stomach cancer are also thought to be linked to smoking along with declines in Helicobacter pylori and improvements in diet [29, 31]. The picture is less clear, given changes in the ways these cancers are coded, but bladder cancer has been shown to be associated with smoking too [32], which may account for the decline in rates. The fall in cervical cancer can be largely attributed to the roll-out of cervical screening since the 1980s, which in addition to detecting cancers, is able to pick up pre-cancerous abnormalities and so reduce the incidence of cancer. This has led to a focus on reduction in smoking and on cervical screening.

In addition to cervical screening (which has been in its present form since 1988), there are currently two other national screening programmes in the UK aimed at early detection of cancers: breast cancer screening (introduced in 1988) and colorectal cancer screening (introduced in 2006). There is evidence that all three programmes have reduced mortality [33–35], and should be included in our list of priority interventions.

There have been substantial advances in cancer treatment in recent decades, which have led to valuable health gains. Surgical techniques remain a cornerstone of treatment, aided by ever-refined radiotherapy methods. The advent of new cytotoxic therapies, as well as hormonal and biological therapies, has greatly increased the available treatment options. Given the breadth of these treatments (and backed up by the number of treatments that expert opinion had identified) it was necessary to limit the focus of our estimation to a subset, which we expected to include most of the health gains likely to have been observed between 1991 and 2010. Data on changes in survival were used as a proxy for health gains. Data were compiled for cancer types on 1-year and 5-year survival rates from CRUK [36] and the Office for National Statistics (ONS) [37] (see Additional file 2). Rates were calculated as percentages for the period 1986 to 1990, and compared with those in 2005 to 2009 to calculate a change in the proportion of people surviving 1 and 5 years after diagnosis. This change in rate was then multiplied by the ‘mid-point’ incidence in 1999 to 2001 to estimate the additional number of people surviving. The same three cancer types (albeit in slightly different order) were found to have the highest number of additional people surviving for both 1 and 5 years; these were prostate, colorectal, and breast cancer. These three accounted for 73% of the estimated gains in 5-year survival. Using clinical guidelines published by the National Institute for Health and Care Excellence (NICE) a set of the main interventions for each of these three cancer types was identified. These interventions were all treatments, because, although there have been improvements in diagnostics and service configuration, it was assumed that the benefits derived from these should, in principle at least, be reflected in the number of people accessing treatment and in measures of treatment effectiveness.

Identifying estimates of per-patient NMB for the set of cancer interventions

As a result of the approach outlined above, estimates of per patient cost and effects were then obtained from published studies for the following prioritised areas:

Smoking prevention/cessation

Screening programmes: cervical, breast, and bowel cancer.

Treatment of: breast, colorectal and prostate cancer.

Smoking prevention/cessation

The area where we adopted a very different approach to that which we had previously used for CVD was smoking. In that study, we restricted the analysis to the costs and benefits arising from NHS smoking cessation interventions. Cancer research has not only unequivocally shown the causal link between smoking (both active and passive) and both cancer and the risks of cancer (and other health problems) but also the effectiveness of various national interventions in reducing smoking rates. This cumulative evidence has contributed to a slow but steady change in smoking behaviour both through direct effects on individual behaviours and through the many non-NHS interventions in the UK (such as legislation and taxation) which have followed from, and been made possible by, this evidence, and have encouraged existing smokers to quit and discouraged others from taking up smoking, as summarised in Figure 1. Therefore, health gains from research include not only the benefit from getting smokers to quit (aided or not by the NHS), but also in preventing non-smokers from ever starting smoking. A recent modelling study for the UK DH Policy Research Programme provided estimates of lifetime life years gained and cost savings to the NHS of non-smokers and ex-smokers compared with smokers [38]. The model accounted for the mortality benefits from not smoking associated with lung cancer, myocardial infarction, stroke, and chronic obstructive pulmonary disease. In the absence of age-specific smoking rates, we used the estimates for men and women aged 35 years, and adjusted these to take account of the proportion of life years gained resulting from lung cancer reduction and also the adjusted life years gained by the population mean utility values for the relevant ages in order to estimate QALYs gained [39].

Figure 1 Smoking behaviour in England, 1982 to 2010. Source: General Lifestyle Survey 2010. The Office for National Statistics. Copyright © 2012, re-used with the permission of The Office for National Statistics. Full size image

Screening programmes

To estimate the NMB of each of the three screening programmes, we identified the most appropriate economic evaluations that modelled the lifetime costs and effectiveness of offering the screening programmes as delivered in the UK. For both cervical and bowel cancer screening we used assessments that had informed relevant screening policy decisions [40, 41]. In the case of cervical screening, we adjusted the figures presented as life years gained by an appropriate age/sex population utility values to give an estimate of QALYs [39]. For breast cancer, we used a recently published economic evaluation that had used a life-table model to assess the overall cost-effectiveness of the NHS screening programme which based its assessment of effectiveness on the findings of the Independent UK Panel on Breast Cancer Screening, and took account of the uncertainty of associated estimates of benefits, harms, and costs [33, 42]. In all three cases, these models used take-up rates that were the same or very similar to those observed in the relevant screening programme during the period in question.

Treatment programmes

The full list of treatment interventions included in the estimation of health gains for each cancer site are shown in Additional file 3. These were determined based on NICE Clinical Guidelines (CG131 for colorectal cancer [43], CG80 and CG81 for breast cancer [44, 45] and CG58 for prostate cancer [46]) and cross-checked to ensure that relevant interventions identified by experts were included. Patient sub-groups were recognised where distinction in treatments was made, or where likely differences in cost and benefits existed. In breast cancer, for instance, this distinction was made for node-positive cancers, oestrogen receptor-positive cancers, HER-2-expressing cancers, and pre/post-menopausal incidence of cancers, and between early-stage and late-stage cancer. Historical comparators for each intervention identified from the contemporary guidelines were then identified back to 1991.

For each of the treatment options considered, published economic evaluations were used to estimate per patient costs and benefits (measured as QALYs). Searches were conducted using the NHS Economic Evaluation Database and MEDLINE to identify economic evaluations of prostate, breast, and colorectal cancer interventions. UK-specific estimates were preferred, but international evidence was used where no appropriate UK estimates were available. Where they were available, NICE technology appraisals and National Institute of Health (NIHR) Health Technology Assessments were used as the most relevant sources (see Additional file 3). Where exceptionally non-UK cost-effectiveness data had to be used, costs were converted using purchasing power parity exchange rates.

Constructing a time series (1991 to 2010) of usage of cancer interventions

To estimate total NMB for the period, per-patient QALY gains and net costs for each intervention were multiplied by the total number of new patients receiving each intervention in each year. We used the following methods to estimate the time series of usage for the selected interventions.

For smoking reduction/cessation we used figures derived from the data on the proportions of smokers, ex-smokers and non-smokers for England for each of the years to estimate the net change per year in QALYs gained and NHS savings achieved, and related these to population data for the UK as a whole [47].

For cervical and breast screening programmes, we used figures for the relevant size of the UK age group in each year to whom screening was first offered (age 25 for cervical and age 50 for breast). For bowel screening we used the numbers first offered screening as the programme began to be rolled out.

To estimate the numbers of people receiving each treatment intervention over time two primary sources were used. For surgical procedures (for example, colorectal excision, liver resection and ablation, prostatectomy, orchiectomy, mastectomy and lumpectomy) Hospital Episodes Statistics [48] were utilised. To estimate the numbers of people receiving drug interventions, data on Net Ingredient Cost (NIC) of drugs to the NHS were used. These data were gathered from Health and Social Care Information Centre (HSCIC) data publications [49], which give details of the total cost of a particular drug prescribed in primary care (for the Prescription Cost Analysis) and secondary care (Hospital Prescriptions Audit Index) in each year. For some drugs, this information was not available for the whole of the time period, in which case assumptions were made on the basis of launch year and the most recent available time point. If the launch year occurred during the period 1991 to 2010, a linear interpolation with launch year at £0 NIC was performed. For drugs that were not launched during the period, a last value carried back approach was adopted, using the most recent year of historical data. From the NIC, the cost and length of a typical regimen (as estimated by NICE costing templates where possible) were used to calculate the number of complete treatments delivered and hence the number of people receiving a particular drug in any given year. This was then proportioned across the indications of a drug and particular patient group (for example, early and late cancers, or multiple cancers).

For some older drug interventions, NIC data were not publicly available for any of the years of interest. In these instances, NICE estimates of the proportion of patients likely to receive interventions (based on guidance costing templates) were combined with data on incidence to estimate usage numbers.

For radiotherapy, there was a paucity of data on usage. Data from the National Clinical Analysis and Specialised Applications Team (NATCANSAT) were available for 2009/10, giving the number of episodes of radiotherapy.e It was estimated that 70% of these episodes would be for primary treatment of a cancer. The number of primary radiotherapy episodes was estimated as a proportion of the incidence of each cancer in 2009/10. This proportion was applied historically to incidence in order to estimate radiotherapy treatment.

The component figures of numbers of people receiving treatment interventions were all derived from data for England. To produce a UK estimate (needed because the research spend data is for the UK) figures were adjusted by a factor reflecting England’s proportion of the adult UK population. The screening was based directly on relevant UK population data, and for smoking behaviour the time series data were for England, but have been applied to the UK population. All cost estimates were adjusted to 2011/12 prices using the Hospital and Community Health Services Pay and Prices Index [50].

For the calculation of NMB, we used for the base case an opportunity cost value of a QALY as used by NICE in its decision-making [51, 52]. This value reflects an estimate of the opportunity cost in terms of QALYs forgone elsewhere in the health service within its fixed budget. Given that public spending on health research can justifiably be seen as a decision to spend on research rather than directly on current healthcare, this opportunity cost value is appropriate to the public decision regarding research funding. In this study, as previously for CVD, we characterised NICE’s threshold range as equivalent to an average of £25,000 per QALY, but considered a broader range of values in the sensitivity analysis, including a value of £70,000, which would be broadly consistent with the commonly proposed QALY threshold of 3 times GDP per capita [53].

Analysis of UK clinical guidelines to estimate elapsed time and rate of attribution

In the 2008 report on CVD research, the references cited in a sample of clinical guidelines were analysed to inform the estimate of the elapsed time between research spend and net health gain, and the proportion of net health gain that could be attributed to UK research [3]. In the current study on cancer research, we replicated this approach.

In total, 31 national clinical guidelines, which provided a broad representation of cancer practice in the UK, were identified. Twelve were published by NICE and a further twelve by the Scottish Intercollegiate Guideline Network (SIGN). The remaining seven guidelines were published by either the Royal Colleges or the National Cancer Screening Programme. The reference sections of these guidelines were reviewed: five had no reference list (four published by NICE, one by the National Screening Programme) while one screening guideline had no references to peer-review journals (that is, it referenced only policy and practice documents). These six guidelines were excluded from our sample. We then used a bespoke computer programme to extract references from the electronic PDF version of each guideline; in three cases the automated reference extraction failed (because papers were not referenced in a recognised format), leaving us with a sample of 22 national guidelines.

Of the 5,627 references cited in the 22 guidelines, 4,416 references (78%) were automatically extracted, excluding duplicate references within a guideline (see Additional file 4 for breakdown by guideline). Nine of these references had no date information and were excluded from the analysis of elapsed time, leaving a total of 4,407 references. The age of a paper cited in a clinical guideline has been termed the ‘knowledge cycle time’ [54], which is the average difference between the publication date of the clinical guideline and the publication date of the cited papers on the guideline. The knowledge cycle time was calculated for the 22 identified guidelines, and used to inform the estimated elapsed time.

To estimate the rate of attribution to the UK, the 4,416 extracted and de-duplicated references were provided to the Centre for Science and Technology Studies (CWTS) to be matched to their bibliometric database (which is derived from the Web of Science).f Of the 4,416 extracted references, CWTS was able to match 4,051 (92%), which formed the dataset to estimate the degree of attribution based on the address field in the cited papers. These addresses were used as a proxy for the location in which the research was conducted, and so it was possible to estimate the proportion of the cited research that was conducted in the UK. The non-matched references included non-serial outputs such as books, journals that are not indexed on the Web of Science, and incorrect references.

Estimation of the rate of return

Using these four key sources of data, we could then attribute a proportion of the estimated total annual NMB of the cancer health gain as being due to UK research, and relate an equal number of years of investment to years of NMB, ‘lagged’ by an estimate of the average lag between research and benefit. The return was expressed as an IRR, which is effectively the discount rate that would yield a zero net present value. The IRR is convenient in enabling a comparison to be made between non-competing investments of different sizes (as well as providing a direct comparison with our previous study). We recognise the many and various layers of estimates involved. In other circumstances, it might be feasible to express the uncertainty as ranges for each parameter in our overall estimate and undertake a formal probabilistic sensitivity analysis (PSA). However, given the nature of the evidence from multiple sources for the numerous parameters and the necessary judgments involved in drawing together and interpreting the evidence, a comprehensive PSA quantitatively characterising all the uncertainty was not feasible here, and indeed would be liable to suggest a spurious precision. Instead we provide a series of one-way and scenario sensitivity analyses to illustrate the effects of specific variables on the IRR.