What every reader should know about studies using electronic health record data but may be afraid to ask

The Consortium For Clinical Characterization Of COVID-19 By EHR (4CE)

Research output: Contribution to journalReview articlepeer-review

27 Scopus citations


Coincident with the tsunami of COVID-19-related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: Data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

Original languageEnglish (US)
Article numbere22219
JournalJournal of medical Internet research
Issue number3
StatePublished - Mar 2021


  • COVID-19
  • Data quality
  • Electronic health records
  • Literature
  • Publishing
  • Quality
  • Real-world data
  • Reporting checklist
  • Reporting standards
  • Review
  • Statistics

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'What every reader should know about studies using electronic health record data but may be afraid to ask'. Together they form a unique fingerprint.

Cite this