Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record

Jennifer A Bierman*, Kathryn Kinner Hufmeyer, David Liss, Charlotta Weaver, Heather L Heiman

*Corresponding author for this work

Research output: Contribution to journalArticle

3 Scopus citations


Construct: We aimed to develop an instrument to measure the quality of inpatient electronic health record– (EHR–) generated progress notes without requiring raters to review the detailed chart or know the patient. Background: Notes written in EHRs have generated criticism for being unnecessarily long and redundant, perpetuating inaccuracy and obscuring providers' clinical reasoning. Available assessment tools either focus on outpatient progress notes or require chart review by raters to develop familiarity with the patient. Approach: We used medical literature, local expert review, and attending focus groups to develop and refine an instrument to evaluate inpatient progress notes. We measured interrater reliability and scored the selected-response elements of the checklist for a sample of 100 progress notes written by PGY-1 trainees on the general medicine service. Results: We developed an instrument with 18 selected-response items and four open-ended items to measure the quality of inpatient progress notes written in the EHR. The mean Cohen's kappa coefficient demonstrated good agreement at.67. The mean note score was 66.9% of maximum possible points (SD = 10.6, range = 34.4%–93.3%). Conclusions: We present validity evidence in the domains of content, internal structure, and response process for a new checklist for rating inpatient progress notes. The scored checklist can be completed in approximately 7 minutes by a rater who is not familiar with the patient and can be done without extensive chart review. We further demonstrate that trainee notes show substantial room for improvement.

Original languageEnglish (US)
Pages (from-to)420-432
Number of pages13
JournalTeaching and Learning in Medicine
Issue number4
StatePublished - Oct 2 2017



  • EHR
  • assessment
  • progress notes
  • trainees

ASJC Scopus subject areas

  • Education

Cite this