The search strategy yielded 37,313 records (Figure 1). After screening for keywords we identified 3137 self-described RCTs. Of these, 83 (2.6%) were excluded on the grounds that they had been published in more than two versions in different journals, leaving 3054 articles selected for author interview. All of the included articles and journals were published in Chinese and peer reviewed. The authors of 735 (24.1%) of these could not be contacted, and a further 84 (2.8%) authors refused to answer our questions. Of the remaining 2235 reports, only 207 were found to be authentic RCTs on the basis of their authors' responses to the interview questions (6.8%, 95% CI 5.9–7.7, in 207 out of 2235 as those studies which the authors cannot be contacted were considered critically as unauthentic RCTs, and 9.3%, 95%CI 8.3–10.3 in 207 out of 3054, respectively). Of these, 103 (7.3%) had examined TCM interventions and 104 (6.4%) had examined CM interventions.

Figure 1 The flow chart used for including and identifying the trials. Full size image

Stratification by type of research setting

An analysis stratified according to type of research setting yielded striking results. Researchers at medical universities or college-affiliated hospitals authored 713 of the self-described RCTs (22.7%). Of these, 30 (4.2%) were found to be the second or third version of a report published in another journal or journals; 162 (23.7%) authors of the original studies could not be contacted, and a further 18 (2.6%) authors refused to participate. The remaining 128 studies were identified as authentic RCTs on the basis of the author interview (18.7%). All of the articles reported results about pre-market drug randomized clinical trials and 51.6% of trials supported by government and other official organizations were identified as authentic.

Authors at level 3 hospitals or medical institutes authored 495 (15.8%) of the self-described RCTs in our sample. Twenty-seven (5.5%) of these were found to be the second or third versions of previously published reports. The original authors of 103 (22.0%) of these reports could not be contacted, and 13 (2.8%) were uncooperative. A total of 55 studies conducted at level 3 hospitals were deemed on the basis of the author interview to be authentic RCTs (11.8%). All of the randomized clinical trials of pre-market drugs were authentic, and of the trials supported by government and other official sources, 56.3% were authentic.

Researchers at level 2 and lower-level hospitals authored 1929 (61.5%) of the self-described RCTs in our sample. Of these, 26 studies (1.3%) had more than two versions. The authors of 470 (24.7%) of the studies from the level 2 group could not be contacted for interview, and 53 (2.8%) refused to cooperate. Only 24 studies conducted at class 2 or lower-level hospitals were deemed authentic RCTs (1.3%); of these, only one was a trial of a pre-market drug, and one had received funding from an official source.

Thus, self-described RCTs conducted at university or college-affiliated hospitals were significantly more likely to be authentic than those conducted at level 3 hospitals and medical institutes (RR 1.58, 95% CI 1.18–2.13) and those conducted at level 2 and lower level hospitals (RR 14.42, 95% CI 9.40–22.10). Similarly, studies conducted at level 3 hospitals were more likely to be authentic than those conducted at level 2 and lower-level hospitals (RR 10.18, 95% CI 6.23–16.63).

Stratification by category of intervention

Of the 713 RCTs conducted at medical-university or college-affiliated hospitals, 331 (22.8%) examined TCM interventions while 382 (22.7%) examined CM interventions. Of these, 11 (3.3%) TCM reports and 19 (5.0%) CM ones were the second or third version of previously published reports. The authors of 75 (23.4%) of the TCM reports and 87 (24.0%) of the CM reports could not be reached, and the authors of 12 (3.8%) of the TCM papers and 6 (1.7%) of the CM papers were uncooperative. Sixty-nine TCM studies (21.6%) and 59 CM studies (16.3%) were considered authentic RCTs; there was no statistically significant difference in rates of authenticity between trials with respect to type of intervention (RR 1.27, 95% CI 0.93–1.74). All of the RCTs of pre-market drugs were authentic, regardless of whether they were classified as TCM or CM. Similarly, there was no statistically significant difference in rates of authenticity between the TCM and CM categories with respect to funding source (RR 1.33, 95% CI 0.97–1.81; P = 0.08 (other source-supported projects), and RR 1.24, 95%CI 0.67 to around 2.30; P = 0.49 (government-supported projects, respectively)). Among the 62 projects funded by government and other official sources, 54.3% of the TCM trials were authentic and 43.8% of the CM were authentic; there was no statistically significant difference in authenticity between TCM and CM fields (RR 1.27, 95% CI 0.93–1.74).

Of the 495 self-described RCTs conducted at level 3 hospitals, 192 (13.2%) involved TCM interventions and 303 (19.1%) CM interventions; only 55 of these 495 studies proved to be authentic RCTs, of which 23 (12.4%) and 32 (11.3%) concerned TCM and CM interventions, respectively. Of these reports, 7 (3.6%) TCM studies and 20 (6.6%) CM studies were the second or third version of previously published reports. The authors of 28 (15.1%) TCM studies and 75 (26.5%) CM studies could not be contacted by telephone, and the authors of 6 (3.2%) TCM studies and 7 (2.5%) CM studies refused to answer our questions.

There was no statistically significant difference in the likelihood of authenticity in the level 3 institutions associated with the category of interventions (RR 1.10, 95% CI 0.67–1.82). Five trials in each of the TCM and CM categories were of pre-market drugs; all were deemed authentic RCTs. The rate of authenticity among government and other officially supported projects was 100% in the TCM category and 30% in the CM category.

Rates of authenticity for different levels of hospital according to category of intervention

Studies conducted at university or college-affiliated hospitals had higher rates of authenticity than studies conducted at class 3 and level 2 hospitals. This difference was statistically significant for both TCM and CM studies (RR 1.68, 95% CI 1.09–2.61, and RR 17.47, 95% CI 9.35–32.64 in the TCM field, and RR 1.46, 95% CI 0.98–2.17, RR 12.53, 95% CI 6.96–22.54 in the CM field, respectively).

Studies conducted at level 3 hospitals and institutes had higher rates of authenticity in comparison with class 2 or lower hospitals. This difference was statistically significant for both TCM and CM studies (RR 11.67, 95% CI 5.58–24.40 for TCM studies, and RR 8.59, 95% CI 4.57–16.15 for CM studies).

Authors' methodological understanding

We found that 115 authors (5.1%) had a good understanding of randomization methods but still claimed that their non-RCTs were RCTs. Of these, 88 (8.2%) had reported on TCM interventions and 27 (2.3%) on CM interventions. We found that 1913 authors (85.6%) did not fully understand the principles of randomization when they incorrectly claimed that their trials were RCTs. Of these 1913 authors, 882 (82.2%) reported on TCM interventions and 1031 (88.7%) on CM interventions. Of course, we cannot judge whether the authors who could not be contacted or refused to answer our questions had a good understanding of randomization methodology.

The results of our stratified analysis are given in Additional file 1. A further result, depicted in Figure 2, was that rates of authenticity (as defined by the use of adequate allocation methodology) among self-described RCTs of both TCM and CM studies have been increasing over the past 10 years.