What Carlisle has come up with is a metric that can be applied to any randomized trial – a single number that might tell you that the data is too suspicious to be believed. But in order to understand the metric, you need to know a few things about randomized trials.

Randomized trials take some population of individuals and then randomize them into two or more groups. Sometimes it is a drug versus a placebo, sometimes a drug vs. a drug, or there could be multiple arms and interventions. The key isn't what is being tested. The key is the word "random".

The magic of randomized trials is that the act of randomization tends to balance baseline variables between the groups. If you want to see how a new cholesterol medication works, you don't want to give all the elderly people the drug and all the young people placebo, you want balance.

There are a variety of ways to randomize, but the hope is that, in the end, the magic of random numbers will do its thing.

And it usually does.

We typically see the effect of randomization in "Table 1" of a medical study. Here's table 1 from my favorite randomized trial: