Understanding Assessment Systems for Clinical Competency Committee Decisions: Evidence from a Multisite Study of Psychiatry Residency Training Programs

R. Brett Lloyd*, Yoon Soo Park, Ara Tekian, Robert Marvin

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


Objective: This multisite study examines how clinical competency committees in Psychiatry synthesize resident assessments to inform milestones decisions to provide guidelines that support their use. Methods: The study convened training directors and associate training directors from three psychiatry residency programs to examine decision-making processes of clinical competency committees. Annual resident assessments for one second year and one third year resident were used in a mock clinical competency committee format to assign milestones for two consecutive reporting periods. The committees reflected on the process and rated how the assessment tools impacted the assessment of milestones and evaluated the overall process. The authors compared reliability of assessment between the mock committees and examined both reliability of end of rotation assessments and their composite scores when combined with clinical skills evaluations. Results: End of rotation evaluations were the most informative tool for assigning milestones and clarifying discrepancies in performance. In particular, the patient care and medical knowledge competencies were the easiest to rate, while the systems-based practice and practice-based learning and improvement were the most difficult. Reliability between committees was low although higher number of available evaluations improved reliability in decision-making. Conclusions: The results indicate that the medical knowledge and patient care competencies are the easiest to rate and informed most by end of rotation evaluations and clinical skills examinations. Other evaluation tools may better capture performance on specific sub-competencies beyond workplace-based assessment, or it may be helpful to reconsider the utility of how individual sub-competencies are evaluated.

Original languageEnglish (US)
Pages (from-to)734-740
Number of pages7
JournalAcademic Psychiatry
Issue number6
StatePublished - Dec 2020


  • Assessment
  • Clinical competency committee
  • Evaluation
  • Milestones

ASJC Scopus subject areas

  • Education
  • Psychiatry and Mental health


Dive into the research topics of 'Understanding Assessment Systems for Clinical Competency Committee Decisions: Evidence from a Multisite Study of Psychiatry Residency Training Programs'. Together they form a unique fingerprint.

Cite this