Effect estimates of COVID-19 non-pharmaceutical interventions are non-robust and highly model-dependent

Vincent Chin, John P.A. Ioannidis*, Martin A. Tanner, Sally Cripps

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

41 Scopus citations


Objective: To compare the inference regarding the effectiveness of the various non-pharmaceutical interventions (NPIs) for COVID-19 obtained from different SIR models. Study design and setting: We explored two models developed by Imperial College that considered only NPIs without accounting for mobility (model 1) or only mobility (model 2), and a model accounting for the combination of mobility and NPIs (model 3). Imperial College applied models 1 and 2 to 11 European countries and to the USA, respectively. We applied these models to 14 European countries (original 11 plus another 3), over two different time horizons. Results: While model 1 found that lockdown was the most effective measure in the original 11 countries, model 2 showed that lockdown had little or no benefit as it was typically introduced at a point when the time-varying reproduction number was already very low. Model 3 found that the simple banning of public events was beneficial, while lockdown had no consistent impact. Based on Bayesian metrics, model 2 was better supported by the data than either model 1 or model 3 for both time horizons. Conclusion: Inferences on effects of NPIs are non-robust and highly sensitive to model specification. In the SIR modeling framework, the impacts of lockdown are uncertain and highly model-dependent.

Original languageEnglish (US)
Pages (from-to)96-132
Number of pages37
JournalJournal of Clinical Epidemiology
StatePublished - Aug 2021


  • Bayesian statistics
  • COVID-19
  • Information criteria
  • Model comparison
  • Non-pharmaceutical interventions
  • SIR models

ASJC Scopus subject areas

  • Epidemiology


Dive into the research topics of 'Effect estimates of COVID-19 non-pharmaceutical interventions are non-robust and highly model-dependent'. Together they form a unique fingerprint.

Cite this