I disagree that the current misuse of P values in biomedical science could be solved by ‘retiring’ statistical significance (V. Amrhein et al. Nature 567, 305–307; 2019).

Like it or not, some of the blame for current practices lies in researchers’ infatuation with simply disproving the null hypothesis. They often see this as a more ‘objective’ way of doing science: collect data and let the decision about its importance be made by statistics.

The real question is whether a treatment effect is important, not whether it differs ‘significantly’ from a control. To answer this, the researcher should justify beforehand how large the effect size needs to be. Then, if a 10% improvement over the control is required, the probability that this has been attained can be calculated from the data using familiar statistical tools for hypothesis testing and sample-size determination.