Development and initial validation of a video-based peroral endoscopic myotomy assessment tool

Dennis Yang*, Peter V. Draganov, Heiko Pohl, Hiroyuki Aihara, Thurarshen Jeyalingam, Mouen Khashab, Nanlong Liu, Muhammad K. Hasan, Salmaan Jawaid, Mohamed Othman, Mohamed Al-Haddad, John M. DeWitt, Joseph R. Triggs, Andrew Y. Wang, Robert Bechara, Amrita Sethi, Ryan Law, Aziz A. Aadam, Nikhil Kumta, Neil SharmaMaham Hayat, Yi Yang Zhang, Fanchao Yi, B. Joseph Elmunzer

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Background and Aims: Video analysis has emerged as a potential strategy for performance assessment and improvement. We aimed to develop a video-based skill assessment tool for peroral endoscopic myotomy (POEM). Methods: POEM was deconstructed into basic procedural components through video analysis by an expert panel. A modified Delphi approach and 2 validation exercises were conducted to refine the POEM assessment tool (POEMAT). Twelve assessors used the final POEMAT version to grade 10 videos. Fully crossed generalizability (G) studies investigated the contributions of assessors, endoscopists’ performance, and technical elements to reliability. G coefficients below .5 were considered unreliable, between .5 and .7 as modestly reliable, and above .7 as indicative of satisfactory reliability. Results: After task deconstruction, discussions, and the modified Delphi process, the final POEMAT comprised 9 technical elements. G analysis showed low variance for endoscopist performance (.8%-24.9%) and high interrater variability (range, 63.2%-90.1%). The G score was moderately reliable (≥.60) for “submucosal tunneling” and “myotomy” and satisfactorily reliable (≥.70) for “active hemostasis” and “mucosal closure.” Conclusions: We developed and established initial content and response process validity evidence for the POEMAT. Future steps include appraisal of the tool using a wider range of POEM videos to establish and improve the discriminative validity of this tool.

Original languageEnglish (US)
Pages (from-to)177-185
Number of pages9
JournalGastrointestinal endoscopy
Volume99
Issue number2
DOIs
StatePublished - Feb 2024

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Gastroenterology

Fingerprint

Dive into the research topics of 'Development and initial validation of a video-based peroral endoscopic myotomy assessment tool'. Together they form a unique fingerprint.

Cite this