Numerous studies have suggested that MDMA can cause neurocognitive deficits. However, the available data can only suggest an association – rather than a causal relationship – between MDMA use and neurocognitive deficits. The reliability and robustness of this association was evaluated using Bradford Hill’s criteria for determining causation in epidemiology research. Several limitations in the literature were found. Studies have recruited people who abuse ecstasy – an illicit drug that does not always contain MDMA. There is inherent risk in consuming impure or falsely identified substances; and using this as a source as for scientific opinion may introduce biases in our understanding the actuals risks associated with MDMA. Importantly, given that ecstasy research is predominately retrospective, baseline functioning cannot be established; which may be influenced by a variety of preexisting factors. Many studies introduce statistical errors by inconsistently dichotomizing and comparing light and heavy ecstasy users, making dose-response relationships inconclusive. When interpreting the ecstasy literature effect sizes are a more meaningful indicator of neurocognitive functioning rather than relying on p-values alone. Most meta-analyses have failed to find clinically relevant differences between ecstasy users and controls. There is also consistent evidence of publication bias in this field of research, which indicates that the literature is both biased and incomplete. Finally, suggestions for improving the ecstasy literature are provided.