Several weeks ago, one of the USA’s better quantitative scientists, John Ioannidis of Stanford, made a critically important point. During the coronavirus pandemic, ‘we are making decisions without reliable data’, he said.

As Ioannidis and others have pointed out, we do not even know the actual death rate for Covid-19. Terrifying and widely cited case-fatality rates like ‘three per cent’ come from comparing known fatalities to the small pool of people who have officially been tested. Those test cases are mostly made up of sick and symptomatic people or those who had direct contact with someone known to have had Covid-19 – rather than to the far larger pool of people who may have had a mild version of the disease. Because of the same denominator problem, we also don’t know the true infection rate. A recent German study indicates this could be as high as 15 per cent. Finally, we do not seem to know the effectiveness of the various strategies adopted by national and regional governments to respond to the disease – ranging from the advocacy of social distancing to full-on lockdowns.

This piece tackles that question. As a professional political scientist, I have analysed data from the Worldometers Coronavirus project, along with information about the population, population density, median income, median age and diversity of each US state, to determine whether states that have adopted lockdowns or ‘shelter in place’ orders experience fewer Covid-19 cases and deaths than those which pursue a social-distancing strategy without a formal lockdown. I then briefly extend this analysis to compare countries. In short, I do not find that lockdowns are a more effective way of handling coronavirus than well-done social-distancing measures. The most basic way to test this thesis is by direct comparison. As of 6 April, seven US states had not adopted shelter-in-place orders, instead imposing social-distancing restrictions such as banning large gatherings and mandating six-foot spacing gaps and maximum customer limits inside all retail stores. Those seven states are Arkansas, Iowa, Nebraska, North Dakota, South Dakota, Utah and Wyoming. These states reported 1,620, 2,141, 952, 343, 1,311, 2,542 and 288 cases of Covid-19 respectively as of 3:40pm EST on 16 April – for an average of 1,321 cases. The states reported 37, 60, 21, 9, 7, 20, and 2 deaths respectively, for an average of 22.3 deaths. Throwing in South Carolina, which did not adopt a shelter-in-place order until 6 April, and still allows most religious services, does not dramatically alter these figures – these states averaged 1,613 cases and 33 deaths.

How do these states measure up to the rest of the US? Rather well. According to Worldometers, by the same time the number of officially tested Covid-19 cases across the US states – including Guam, Puerto Rico, and Washington, DC – ranged from 226,343 in New York to 135 in Guam. The average number of Covid cases in a US state was 12,520. The state-by-state number of deaths varied from 16,251 (New York) to two (Wyoming), with the average figure for deaths being 642. Removing the outlier case of New York state, where roughly half of all US Covid-19 deaths have taken place, shifted these figures downward somewhat – to 8,408 cases and 342 deaths in the average state. However, the social-distancing states experienced substantially fewer cases and deaths than the lockdown states, even with New York out of the mix. An advocate of lockdowns could object that the social-distancing states are little places, located in America’s ‘flyover land’. While this charge might be based as much on bias as reality – Utah, Nebraska and South Carolina are sizable places – the next step of my analysis was to adjust for population, using a standard deaths-per-million metric. In alphabetical order, the seven social-distancing states experienced 12, 19, 11, 12, 8, 7 and three deaths per million – for an average of 10 deaths per million when you exclude South Carolina and 12 with South Carolina included.

Again, these numbers compare very favourably to the US as a whole, despite adjusting for population. Across all US states, the number of deaths per million varied from 828 (New York) to three (Wyoming), for an average of 69. With New York removed from the mix, the hardest-hit remaining state was New Jersey, with 8,480 cases and 396 deaths. The average number of cases-per-million across the states minus New York was 1,392 and the average number of deaths-per-million was 54. Comparing the social-distancing states plus South Carolina to US states minus New York, the social-distancing states experienced 663 fewer cases per million and 42 fewer deaths per million on average than the lockdown states. Next, I ran a regression model. For those unfamiliar with academic statistical methods, regression – in this case linear regression – is a computerised mathematical technique that allows researchers to measure the influence of one variable on another with all of the other factors that might be relevant held constant. In this case, the variables for each state included in my model were: population, population density, median income, median age, diversity (measured as the percentage of minorities in a population), and the state’s Covid-19 response strategy (0 = lockdown, 1 = social distancing). The data set used to construct this model is available for anyone to request it.

The question the model set out to ask was whether lockdown states experience fewer Covid-19 cases and deaths than social-distancing states, adjusted for all of the above variables. The answer? No. The impact of state-response strategy on both my cases and deaths measures was utterly insignificant. The ‘p-value’ for the variable representing strategy was 0.94 when it was regressed against the deaths metric, which means there is a 94 per cent chance that any relationship between the different measures and Covid-19 deaths was the result of pure random chance. The only variable to be statistically significant in terms of cases and deaths was population (p=0.006 and 0.021 respectively). Across the US states, each increase in the population of 100,000 correlated with 1,779 additional Covid-19 cases, even with multiple other factors adjusted for. Large, densely populated areas are more likely to struggle with Covid-19, no matter what response strategy they adopt – although erring on the side of caution might make sense for global megacities such as New York and Chicago.

Finally, I extended my analysis into the international arena. As has been widely reported, Sweden has opted not to lock down in the wake of Covid-19, and Swedes have instead followed similar social-distancing measures to those adopted in the seven US states I focused on. Again, there is very little evidence that Sweden has become an unlivable Covid-19 hotbed. As of 17 April, Sweden’s Covid-19 statistics were: 13,216 total cases, 1,400 total deaths, 1,309 cases per million and 139 deaths per million. In terms of cases per million residents, Sweden ranks slightly ahead of its close neighbours, Denmark (1,221) and Norway (1,274). But in Europe as a whole, Sweden ranks 23rd in terms of cases per million and 10th in terms of deaths per million.

I am reluctant to compare European examples to the many East Asian countries which avoided significant shutdowns – particularly since these countries had significantly better early-response strategies and there can be larger cultural differences which are difficult to quantify. But essentially, the same pattern holds true. When I conducted my analysis, Japan had 9,231 total cases, 190 total deaths, 73 cases per million citizens, and two deaths per million. South Korea had 10,635 cases, 230 deaths, 207 cases per million and four deaths per million. Taiwan had a total of 395 cases and only six deaths, alongside 17 cases per million and 0.03 deaths per million. Of course, no single analysis can provide a truly conclusive answer to questions as huge as those posed by Covid-19. Scholars and curious citizens reading this one might want to re-run my analysis with current active cases as a dependent variable rather than total cases or cases per million – although I doubt that would make much difference. It certainly might make sense to redo my regression with ‘date of first case’ thrown in as a variable. I kept the model limited to five independent variables due to the small number of state-level observations available, and left that one out because onset dates were fairly similar for most US states. However, including this information could theoretically produce different results. The more data, the better.

Overall, however, the fact that good-sized regions from Utah to Sweden to much of East Asia have avoided harsh lockdowns without being overrun by Covid-19 is notable. The original response to Covid-19 was driven by an understandable fear of an unknown disease. The epidemiologist Neil Ferguson projected that 2.2million people could die in the US alone, and few world leaders were willing to risk being the one who would allow such grim reaping to occur.

However, as time has passed, new data have emerged. A top-quality team from Stanford University has pointed out that the infection rate for Covid-19 must logically be far higher than the official tested rate, and the fatality rate for the virus could thus be much closer to 0.1 per cent than the 2 to 4 per cent that was initially expected. And empirical analyses of national and regional response strategies, including this one, do not necessarily find that costly lockdowns work better against the virus than social distancing. It should not be taboo to discuss these facts.