Registration of SRs in PROSPERO has increased rapidly, from 285 records in the year of inception (i.e. 2011) to 30,000 by the end of 2017. Half a million visits to the website occurred in the last 12 months from all over the world, most commonly from the UK. In a random sample of 150 SRs registered in PROSPERO between April 1, 2017 and September 30, 2017, the majority were focused on a therapeutic or epidemiologic question, and are being conducted without a dedicated funding source. A wide range of health areas were addressed across the 150 SRs, the most common being diseases of the circulatory system, mental and behavioural disorders and diseases of the musculoskeletal system and connective tissue. Pre-specification of primary outcomes was incomplete in most of the 150 records, with almost a third specifying the domain only.

Strengths and limitations

A strength of our study is that the PROSPERO database managers provided us with routinely collected data on the number of PROSPERO registrations and website usage, which removed the potential for errors due to manual data collection. Further, we were able to explore trends in these variables over time, unlike a previous evaluation which was limited to the first year of activity [13]. However, there are also some limitations. We analysed other variables (epidemiological characteristics and completeness of outcome pre-specification) in a random sample of SRs registered in PROSPERO between April and September 2017 only. Therefore, our frequency statistics may not generalise to SRs registered earlier. It is possible that some errors exist in our data on epidemiological characteristics and completeness of outcome pre-specification, given that data collection and classification were performed by one author only. However, we expect the number of errors to be low given the extensive experience that the data collector (MJP) has from recording this type of information in previous studies [1, 14, 17, 22].

Comparison with other studies

Compared with a previous analysis of the characteristics of PROSPERO registrations [13], the geographical scope of SRs registered has changed considerably over time. Registrations in the first year (2011–2012) came from 33 different countries, whereas that figure has risen to over 100 countries now. England remains the country responsible for most registrations, and Italy has moved into the list of top ten contributors, overtaking Denmark. The number of registrations submitted by authors based in Australia and China has also increased; in the first year of operation, these countries were in fourth and eighth position on the list of top contributing countries [13], and have moved into second and fourth position, respectively.

Explanations and implications

The exponential increase in SR registrations from 2011 to 2017 is a surprising, albeit welcome, development. Unlike for clinical trialists, who rapidly embraced trial registration only after the International Committee of Medical Journal Editors (ICMJE) announced that they would no longer publish trials that were not registered at inception [23], SR registration is not yet required by most journals (nor is it an ICMJE journal requirement). Therefore, the motivation of systematic reviewers to register their SR is likely due to other factors. It is possible that authors are paying attention to the increasing number of publications promoting the advantages of pre-registration of scientific studies and open science practices [24,25,26]. The uptake of registration could also be driven in part by journals that endorse the PRISMA Statement, which encourages SR registration in item 5 [6]. In addition, awareness of the benefits of SR registration likely increased following the dissemination of the PRISMA for Protocols Statement in January 2015 [27, 28], which mentions PROSPERO specifically in item 2. Further, many authors of SRs may be trialists as well, so an acceptance of the need for trial registration may have translated to an acceptance of SR registration.

The epidemiological characteristics of the registered SRs we examined share some similarities, and some differences, with a recent sample of completed SRs [1]. In both samples, most of the SRs focused on a therapeutic question (55% of 300 SRs indexed in MEDLINE® in February 2014 versus 52% of 150 SRs registered in PROSPERO in 2017), and few focused on a diagnostic/prognostic question (11 versus 7%, respectively). This possibly reflects the fact that methods for therapeutic SRs are more established than methods for other types of SRs [29, 30]. Diseases of the circulatory system was one of the most common health areas addressed in both samples; a positive sign given that conditions such as ischaemic heart disease and stroke are leading causes of mortality globally [31, 32]. Neoplasms and certain infectious and parasitic diseases—other leading causes of mortality [32]—were the most common areas addressed in the SRs indexed in MEDLINE® in February 2014 (in 16 and 14% of 300 SRs, respectively), yet both diseases were addressed in fewer (7%) of the 150 SRs registered in PROSPERO in 2017. We encourage systematic reviewers planning SRs to align their topic with global burden of disease data to ensure they are targeting clinically important questions [33].

None of the 150 registered SRs that we studied in detail were funded by a for-profit source. This is concerning given that other research has revealed a proliferation of meta-analyses are being commissioned by industry, most of which are not registered or published [34]. Clearly, more work is needed to encourage for-profit companies to embrace SR transparency practices. In the future, it would be worthwhile to explore (e.g. via surveys and interviews) the reasons for non-registration of SRs by investigators working in for-profit and non-profit organisations.

The incomplete pre-specification of outcomes that we observed in PROSPERO records is consistent with that observed in previous studies evaluating outcomes in Cochrane SR protocols [17, 18] and published SRs [17, 20]. This is likely because full pre-specification of outcomes is not yet a mandatory requirement for SR registration. We believe this should be reconsidered. There is emerging evidence that multiple results are often available for the same outcome in clinical trials (e.g. pain is measured using three scales, each at two time points), and that failure to completely pre-specify SR outcomes can lead to challenges with selecting and interpreting results from the included trials [17, 35, 36]. For example, Mayo-Wilson et al. found that across 21 trials of gabapentin for neuropathic pain, the number of unique within-trial results for pain intensity that systematic reviewers could select from ranged from 1 to 68 (because of within-trial multiplicity of outcome measurements, metrics, methods of aggregation and time points). For this reason, the total number of possible meta-analyses for pain intensity that systematic reviewers could (in theory) calculate was more than 34 trillion [36]. To deal with multiplicity challenges such as these, and prevent cherry-picking of the most favourable results when multiple are available within studies, systematic reviewers should make greater use of the “Timing and effect measures” field in PROSPERO, to completely pre-specify the outcomes of interest to the SR. In addition, such information in PROSPERO records should be accurately reflected in public versions of SR protocols and vice versa, and any amendments should be documented in completed SR articles.

There are several avenues of further work relating to PROSPERO. The field of knowledge synthesis is evolving to include other types of reviews, such as scoping reviews [37], living SRs [38] and rapid reviews [39]. Given the rapidly changing environment and methods, PROSPERO may need to adapt the criteria for inclusion in the database, as well as items on the registration form, to reflect the current state of the field. Also, it would be worthwhile to conduct a large-scale investigation of publication rates of registered SRs, the prevalence of and reasons for discontinued SRs and discrepancies in the planned and reported SR methods. Finally, that 30,000 SRs are underway suggest that the mass production of SRs previously discussed [40, 41] shows no sign of waning. It remains to be seen whether all of these SRs address questions that are relevant to end-users (e.g. patients, health care providers and policy makers), are well conducted, and are free of financial conflicts of interest. The PROSPERO management team have advised us that to minimise the potential for redundancy of SRs, forthcoming changes to the user interface will require those registering SRs to consider whether similar SRs already exist, and whether a new SR is necessary (L. Stewart, personal communication).