E-learning teaches attendings “how to” objectively assess pediatric urology trainees' surgery skills for orchiopexy

Nicolas Fernandez, Max Maizels*, Walid Farhat, Edwin Smith, Dennis B Liu, Michael Chua, Yasin Bhanji

*Corresponding author for this work

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Introduction: Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. Methods: An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). Results: A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. “Core concepts,” in which users learned the assessment tool methods; 2. “Learn to assess,” in which users learned how to assess by watching video clips, explaining the assessment method; and 3. “Test,” in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure). A total of 89 surgical skill ratings were performed with 56 (65%) exact matches between raters and 89 (100%) matched within one rank. Interclass correlation coefficient (ANOVA) showed statistically significant correlation. (r = 0.725, 95% CI 0.571–0.837, F = 3.976, p ≤ 0.00001). Kappa analysis of inter-rater reliability showed strong consensus between attendings for average measures with ICC = 0.71, 95% CI 0.46–0.95 (p = 0.03). Conclusion: We launched e-learning to teach pediatric urology attendings “how to” assess trainee surgical skills objectively (Zwisch scale). After e-learning, there was strong inter-rater correlation in assessments made. We plan to extend such e-learning to pediatric urology surgical training programs.[Figure presented]

Original languageEnglish (US)
Pages (from-to)132.e1-132.e6
JournalJournal of Pediatric Urology
Volume14
Issue number2
DOIs
StatePublished - Apr 1 2018

Fingerprint

Orchiopexy
Urology
Learning
Pediatrics
Surgical Instruments
Videotape Recording
Education
Aptitude
Groin
Operating Rooms
Internship and Residency
Consensus
Analysis of Variance
Teaching

Keywords

  • CEVL
  • Orchidopexy
  • Surgery education

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Urology

Cite this

Fernandez, Nicolas ; Maizels, Max ; Farhat, Walid ; Smith, Edwin ; Liu, Dennis B ; Chua, Michael ; Bhanji, Yasin. / E-learning teaches attendings “how to” objectively assess pediatric urology trainees' surgery skills for orchiopexy. In: Journal of Pediatric Urology. 2018 ; Vol. 14, No. 2. pp. 132.e1-132.e6.
@article{3ff24005d04e40df82bf42afb20922f6,
title = "E-learning teaches attendings “how to” objectively assess pediatric urology trainees' surgery skills for orchiopexy",
abstract = "Introduction: Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. Methods: An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). Results: A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. “Core concepts,” in which users learned the assessment tool methods; 2. “Learn to assess,” in which users learned how to assess by watching video clips, explaining the assessment method; and 3. “Test,” in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure). A total of 89 surgical skill ratings were performed with 56 (65{\%}) exact matches between raters and 89 (100{\%}) matched within one rank. Interclass correlation coefficient (ANOVA) showed statistically significant correlation. (r = 0.725, 95{\%} CI 0.571–0.837, F = 3.976, p ≤ 0.00001). Kappa analysis of inter-rater reliability showed strong consensus between attendings for average measures with ICC = 0.71, 95{\%} CI 0.46–0.95 (p = 0.03). Conclusion: We launched e-learning to teach pediatric urology attendings “how to” assess trainee surgical skills objectively (Zwisch scale). After e-learning, there was strong inter-rater correlation in assessments made. We plan to extend such e-learning to pediatric urology surgical training programs.[Figure presented]",
keywords = "CEVL, Orchidopexy, Surgery education",
author = "Nicolas Fernandez and Max Maizels and Walid Farhat and Edwin Smith and Liu, {Dennis B} and Michael Chua and Yasin Bhanji",
year = "2018",
month = "4",
day = "1",
doi = "10.1016/j.jpurol.2017.09.026",
language = "English (US)",
volume = "14",
pages = "132.e1--132.e6",
journal = "Journal of Pediatric Urology",
issn = "1477-5131",
publisher = "Elsevier BV",
number = "2",

}

E-learning teaches attendings “how to” objectively assess pediatric urology trainees' surgery skills for orchiopexy. / Fernandez, Nicolas; Maizels, Max; Farhat, Walid; Smith, Edwin; Liu, Dennis B; Chua, Michael; Bhanji, Yasin.

In: Journal of Pediatric Urology, Vol. 14, No. 2, 01.04.2018, p. 132.e1-132.e6.

Research output: Contribution to journalArticle

TY - JOUR

T1 - E-learning teaches attendings “how to” objectively assess pediatric urology trainees' surgery skills for orchiopexy

AU - Fernandez, Nicolas

AU - Maizels, Max

AU - Farhat, Walid

AU - Smith, Edwin

AU - Liu, Dennis B

AU - Chua, Michael

AU - Bhanji, Yasin

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Introduction: Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. Methods: An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). Results: A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. “Core concepts,” in which users learned the assessment tool methods; 2. “Learn to assess,” in which users learned how to assess by watching video clips, explaining the assessment method; and 3. “Test,” in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure). A total of 89 surgical skill ratings were performed with 56 (65%) exact matches between raters and 89 (100%) matched within one rank. Interclass correlation coefficient (ANOVA) showed statistically significant correlation. (r = 0.725, 95% CI 0.571–0.837, F = 3.976, p ≤ 0.00001). Kappa analysis of inter-rater reliability showed strong consensus between attendings for average measures with ICC = 0.71, 95% CI 0.46–0.95 (p = 0.03). Conclusion: We launched e-learning to teach pediatric urology attendings “how to” assess trainee surgical skills objectively (Zwisch scale). After e-learning, there was strong inter-rater correlation in assessments made. We plan to extend such e-learning to pediatric urology surgical training programs.[Figure presented]

AB - Introduction: Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. Methods: An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). Results: A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. “Core concepts,” in which users learned the assessment tool methods; 2. “Learn to assess,” in which users learned how to assess by watching video clips, explaining the assessment method; and 3. “Test,” in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure). A total of 89 surgical skill ratings were performed with 56 (65%) exact matches between raters and 89 (100%) matched within one rank. Interclass correlation coefficient (ANOVA) showed statistically significant correlation. (r = 0.725, 95% CI 0.571–0.837, F = 3.976, p ≤ 0.00001). Kappa analysis of inter-rater reliability showed strong consensus between attendings for average measures with ICC = 0.71, 95% CI 0.46–0.95 (p = 0.03). Conclusion: We launched e-learning to teach pediatric urology attendings “how to” assess trainee surgical skills objectively (Zwisch scale). After e-learning, there was strong inter-rater correlation in assessments made. We plan to extend such e-learning to pediatric urology surgical training programs.[Figure presented]

KW - CEVL

KW - Orchidopexy

KW - Surgery education

UR - http://www.scopus.com/inward/record.url?scp=85034845120&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034845120&partnerID=8YFLogxK

U2 - 10.1016/j.jpurol.2017.09.026

DO - 10.1016/j.jpurol.2017.09.026

M3 - Article

VL - 14

SP - 132.e1-132.e6

JO - Journal of Pediatric Urology

JF - Journal of Pediatric Urology

SN - 1477-5131

IS - 2

ER -