The public health response to the new coronavirus continues to evolve rapidly, with states shutting down schools, restaurants, bars, vacation places, and even elections. At the same time, the nation’s capacity to test individuals continues to steadily ramp up. This has led some people asking, “Why don’t we do what South Korea does and just test anyone?”

Test-kit availability aside, there are crucial issues to consider. For example, so long as the background level of infection is low, there are real downsides to mass testing, and good reasons to limit testing to individuals who show symptoms or have been in contact with people who have shown symptoms. The problem is that when the overall level of infection is low, the overwhelming majority of your positive test results from mass testing will be false positives. This gives the public a false sense of what the actual mortality level is, a false sense of security in their own immunity status, and can contribute to future outbreaks. In fact, the mass testing in South Korea could be skewing their data.

To see why this might be the case, I draw upon this thread from Dr. Sterling Haring at Vanderbilt University, as well as my own statistical background. Like him, rather than walking through the actual math of Bayes Rule (I explore it here), I utilize 2x2 charts. We’ll start with the claim from Ohio Gov. Mike DeWine last Thursday that 100,000 people in Ohio were infected. That seems high, but let’s take it. That works out to a little less than 1% of the state’s population having the virus. So we have a society that looks something like this:

What happens if everyone is tested? If tests were perfect, this would be great. But almost all of them come with errors. My understanding is that the quick test used in South Korean drive-throughs generates an error roughly one in 10 times (which would still be much more accurate than the rapid flu tests), while the test with the longer turnaround generates an error roughly one in 20 times.

Let’s assume we tested everyone with the 90% accurate test. We would get a result that looks something like this:

This is not bad. Most of the people who have the virus get a positive reading. What about the people who don’t have the illness?

Most of them get a negative reading. The problem is that, since there are far more people that don’t have the virus than do have it, the 10% error rate for that group overwhelms the 90% accuracy rate for the group that does have it. You end up with a scenario where 93% of the people who test positive for the disease do not, in fact, have it.

What’s the downside of false positives? There are a couple. First, it can skew your data. A number of people have looked at South Korea’s findings and noted the relatively low mortality rate – dropping below 1%. The problem is that with widespread testing, a lot of people who tested positive there won’t actually have the disease in the first place. The virus will be less widespread than the data suggest, but also deadlier.

Second, it can give people a false sense of confidence. We don’t know whether you can get the disease twice, and there are at least two strains of the virus floating around out there. The question is one of public perception. If people believe that you can only get it once – and the possibilities for disinformation on the Internet are legendary – and go out falsely believing that they are immune, then they are susceptible to actual infection from the people who had the sickness and falsely believed they did not have it. They can also conclude that they must have the flu or a cold and delay seeking medical treatment.

What if we use the quick test for screening, and tell people that they need a follow-up if they get a positive reading? Giving the more accurate test to the subset of people who tested positive the first time around is useful – but the false positive rate is still 40%. And our false negative rate starts to creep up as well, with 15% of the people with the disease now getting a medical “all clear.”

To be clear, none of this is meant to suggest that we shouldn’t test at all. It is simply to say that testing isn’t the panacea that many are hoping it is. After all, South Korea may have widely available drive-through testing, but it also pioneered social distancing and did extensive tracking of contacts with infected people. It is also important to emphasize that our scenario assumes the disease has not become widespread; if 5% of the population is infected, the false-positive rate from the second test plummets to just 10%. If we look only at the population that exhibits symptoms, it would be even lower.

All this points toward a larger medium-term problem. The social distancing measures being implemented will likely result in fewer daily infections but also come at great societal costs and are likely untenable for more than a few months, if that. With an estimated 12 to 18 months to go until a vaccine is available, the relaxing of social distancing measures is likely not the beginning of the end, but the end of the beginning. More accurate testing may help, but absent that, we’re likely in for a bumpy ride.