We know some very important facts about COVID-19, including that vaccines work well. I do not think we understand as much as we should about the effects of government policies on the pandemic. For instance, any medical professional would wear personal protective gear while treating a patient with COVID, but it is less clear that requiring a population to wear masks has any effect. There is often a big slippage between voluntary behavior by trained professionals and large-scale mandates.

I did my own light modeling last April and found no effects of state masking and vaccination mandates on COVID mortality rates. I did find that COVID deaths by US state reflected the percentage of the population who had been in poor health before the pandemic, GOP vote share (Trump support meant higher death rates), Black/White segregation, economic inequality, the percent of the population over age 65, the incarceration rate, and a lower college graduation rate. (Statistically significant correlates: the first three. Adjusted r-square of the whole model = .699).

A new paper by Sun & Biseti examines the effects of state policies on county-level COVID death rates during the first 39 weeks of the pandemic, i.e., before vaccines were available and before masks were being widely recommended. They create a “stringency index” composed of “closures of schools, closures of workplaces, cancellations of public events, restrictions on gatherings, closures of public transport, stay-at-home orders, restrictions on domestic movement, and restrictions on international travel.”

Their model incorporates some similar contextual factors to mine but assesses different policies. It suggests that if every state had employed the maximum of all the stringent measures, the national death rate would have fallen by about 7% in 2020, but the benefits would have been greater “in counties with fewer physicians and larger shares of older adults, low-educated residents, and Trump voters” as well as “in rural areas and counties with higher social capital and larger shares of uninsured residents,” while the benefits would have been smaller “in counties with larger shares of [non-Hispanic] Black and Hispanic residents.” Although I don’t think we can tell from their model itself, it’s plausible that closing schools and businesses had no effect on deaths from the disease in big cities, although the closures were very hard on people.

An older but still valuable article (Sharma, Mindermann & Rogers-Smith 2021) looks at similar measures across subnational units (such as regions or states) of European countries. Their model finds significant benefits from stringent measures such as school closings, but smaller benefits during the pandemic’s second wave than its first. That finding illustrates that we are not in the domain of scientific laws here; we are in a messy zone of rapid change.

In my view, democratic governments and other legitimate institutions have a right to impose many kinds of restrictions to combat a disease. They must do their best to make decisions in the face of uncertainty and conflicting interests. No one’s fundamental human rights have been violated if a government closes schools for months or makes one wear a mask during a pandemic.

On the other hand, this does not mean that the most stringent measures are always effective or that we should be overly confident that we know what works. On the contrary, the lessons of this pandemic appear murky to me, and humility is warranted by all.

A lot of people are very sure what should have been done and are certain their opponents are badly motivated or fools. I think most of us did our best and still don’t have a firm basis to know what we should do next time.

Sources: Sharma, M., Mindermann, S., Rogers-Smith, C. et al. Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe. Nat Commun 12, 5820 (2021). https://doi.org/10.1038/s41467-021-26013-4; Sun, Y., & Bisesti, E. M. (2023). Political Economy of the COVID-19 Pandemic: How State Policies Shape County-Level Disparities in COVID-19 Deaths. Socius, 9, 23780231221149902. See also: what explains state variation in COVID-19 mortality?; we must be able to disagree about pandemic policies; vaccination, masking, political polarization, and the authority of science.