The main finding of this systematic review was that the study design used with the majority of studies investigating the ergogenic effect of carbohydrates ingested in the proximity of or during a performance bout was not suitable for extrapolating the performance outcome to a real-life situation. Further, with half of the eligible studies, either a carboloading regime during the days prior to, or a carbohydrate intake in the proximity of or during a performance test did not significantly improve the TT performance or TT part of an S+TT. It is also noteworthy that no study had subjects with a mean VO 2max that would classify them as elite endurance athletes at a high international level (about 70 to 80 mL/kg body mass/min [27]). Studies in which women served as subjects were non existing, except for one study where both genders made up the study population.

Development of recommendations

Systematic reviews of the scientific literature are considered a scientific and ethical imperative when developing policies and practical recommendations [28]. Nevertheless, systematic reviews are unfortunately not always an integral part of such processes. The probably most prominent negative example was the World Health Organization, where "Systematic reviews and concise summaries of findings [were] rarely used for developing recommendations. Instead, processes usually [relied] heavily on experts…"[29]. Relying on experts when developing recommendations is a common procedure and not necessarily problematic. It only becomes a problem if the experts either do not use systematic approaches or do not properly describe the methodology used. Regrettably, the latter often seems to be the case [30]. For example, during the process of performing this review we have encountered articles with missing information on the gender, age, exercise history, or regular training load of the subjects, lacking information on the blinding or randomization of the intervention, and missing information on a pre-exercise meal intake or on the amount of fluids ingested during the performance tests.

Definition of eligibility criteria

The omission of methodological information in systematic reviews is not the only issue with such analyses. The definition of the inclusion/exclusion criteria is a further crucial step determining the validity of a systematic review [31]. However, no universal set of eligibility criteria exists, as the criteria must be fit for the specific purpose of the review. According to our focus on the real-life portability of the performance outcome, we have used trained subjects exercising in a postprandial state and a performance test similar to a competitive event as main criteria.

These criteria have ultimately led to the exclusion of quite a number of studies, and one could argue that the criteria, therefore, were too restrictive. However, performing in a postprandial compared to a fasted state might indeed be two different things related to the potential ergogenic effect of carbohydrates. In a sub-analysis of the meta-analysis by Temesi et al. [7], the influence of carbohydrate ingestion during endurance performance was investigated according to the fasting state (> 8-h-fasted subjects vs. < 6-h-fasted subjects). The reported effect size of the intervention was similar and significant for both fasting states in the case of S+TT interventions but differed in the case of TT interventions (significant only for the < 6-h-fasted subjects). This result is actually not in line with our review, as we have observed rather mixed results with subjects who were not fasted (although our criteria was 4 to 2 h and not < 6 h). Nevertheless, it indicates that the prandial state might influence a nutritional intervention, and therefore one should consider it with performance studies.

Attempt in identifying all relevant studies

A second major aspect to consider when conducting a systematic review is the identification of possibly all studies fit for the purpose of the review [31]. A search across multiple databases seems by nature to be more promising in locating more relevant articles than restricting the search to one database only. As we have only searched the PubMed, we may have missed a substantial number of relevant studies, which would have biased our results.

The comparison with the studies identified for the meta-analysis by Temesi et al. [7] in which seven additional scientific databases were searched yielding a total of 41,175 references (compared to 16,658 in our case) and in which less restricting eligibility criteria were applied, led to the identification of only one additional article [9] eligible for our review. The comparison with the meta-analysis by Vandenbogaerde and Hopkins [8], who used Google Scholar for their search, led to the additional identification of the same additional article already identified by the comparison of the meta-analysis by Temesi et al. In contrast, we identified eight articles [10–14, 19, 21, 25] that were not included in the meta-analysis by Temesi et al. [7] and eight articles [10–12, 18, 19, 22, 23, 25] that were not included in the meta-analysis by Vandenbogaerde and Hopkins [8]. Overall, this indicates that in spite of searching only one database, we likely did not miss a substantial part of published studies on the topic under investigation.

Concluding remarks

The current consensus indicates that carbohydrates ingested in the proximity of or during a performance bout are ergogenic. However, the application of rigorous criteria to a systematic review, such as excluding fasted subjects and time-to-exhaustion test modes, led to a less convincing picture. We observed no significant performance improvement with most of the performance bouts lasting less than 70 min, and the results with longer performance bouts indicated a significant improvement with 10 of 17 interventions.

The absence of clear evidence is, nevertheless, not clear evidence of an absent effect. This is particularly true for the present review as we discarded many studies because relevant information was missing in the articles. As mentioned above, we encountered studies among other with missing information on age, gender, prandial state, or VO 2max of the subjects, missing information on the blinding or randomization of the interventions, or missing information on the drink volume ingested during the intervention. Thus, we cannot exclude that some or even all of these discarded studies would have met the inclusion criteria if only the description were appropriate, and that then the outcome would have been a different one.

The omission of properly describing the methodological part and the frequent use of study designs not allowing an extrapolation of the results to real-life situations are the main reason why it is now difficult to draw a solid conclusion about the potential ergogenic effect of carbohydrates ingested in the proximity of or during a performance bout. Further, it is possible that some of the non-significant studies suffered from low statistical power, particularly studies with just 7 to 8 subjects. Being conservative, however, we can state that with shorter duration events up to perhaps 70 min an ergogenic effect of carbohydrates ingested in the proximity of or during a performance bout is unlikely with trained (but not elite) male athletes in a real-life competition. The picture for longer durations is slightly more in favor than against the current consensus, but it is too heterogeneous for a solid conclusion. Definitely, there is a need for more and more comprehensively described studies to enlighten the current picture.