Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment

Jonathan Paul Fryer, Ezra Nathaniel Teitelbaum, Brian C. George, Mary C. Schuller, Shari Lynn Meyerson, Christina M. Theodorou, Joseph Kang, Amy Yang, Lihui Zhao, Debra DaRosa

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Objective: We have previously demonstrated the feasibility and validity of a smartphone-based system called Procedural Autonomy and Supervision System (PASS), which uses the Zwisch autonomy scale to facilitate assessment of the operative performances of surgical residents and promote progressive autonomy. To determine whether the use of PASS in a general surgery residency program is associated with any negative consequences, we tested the null hypothesis that PASS implementation at our institution would not negatively affect resident or faculty satisfaction in the operating room (OR) nor increase mean OR times for cases performed together by residents and faculty. Methods: Mean OR times were obtained from the electronic medical record at Northwestern Memorial Hospital for the 20 procedures most commonly performed by faculty members with residents before and after PASS implementation. OR times were compared via two-sample t-test. The OR Educational Environment Measure tool was used to assess OR satisfaction with all clinically active general surgery residents (n = 31) and full-time general surgery faculty members (n = 27) before and after PASS implementation. Results were compared using the Mann-Whitney rank sum test. Results: A significant prolongation in mean OR time between control and study period was found for only 1 of the 20 operative procedures performed at least 20 times by participating faculty members with residents. Based on the overall survey score, no significant differences were found between resident and faculty responses to the OR Educational Environment Measure survey before and after PASS implementation. When individual survey items were compared, while no differences were found with resident responses, differences were noted with faculty responses for 7 of the 35 items addressed although after Bonferroni correction none of these differences remained significant. Conclusions: Our data suggest that PASS does not increase mean OR times for the most commonly performed procedures. Resident OR satisfaction did not significantly change during PASS implementation, whereas some changes in faculty satisfaction were noted suggesting that PASS implementation may have had some negative effect with them. Although the effect on faculty satisfaction clearly requires further investigation, our findings support that use of an autonomy-based OR performance assessment system such as PASS does not appear to have a major negative influence on OR times nor OR satisfaction.

Original languageEnglish (US)
Pages (from-to)333-343
Number of pages11
JournalJournal of Surgical Education
Volume75
Issue number2
DOIs
StatePublished - Mar 1 2018

Fingerprint

Operating Rooms
autonomy
resident
supervision
surgery
Electronic Health Records
performance assessment
Operative Surgical Procedures
Internship and Residency
Nonparametric Statistics
memorial
time
electronics

Keywords

  • autonomy
  • evaluation
  • graduate medical education
  • surgery

ASJC Scopus subject areas

  • Surgery
  • Education

Cite this

Fryer, Jonathan Paul ; Teitelbaum, Ezra Nathaniel ; George, Brian C. ; Schuller, Mary C. ; Meyerson, Shari Lynn ; Theodorou, Christina M. ; Kang, Joseph ; Yang, Amy ; Zhao, Lihui ; DaRosa, Debra. / Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment. In: Journal of Surgical Education. 2018 ; Vol. 75, No. 2. pp. 333-343.
@article{11496749cf4b45fd94044d61b0bc1d58,
title = "Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment",
abstract = "Objective: We have previously demonstrated the feasibility and validity of a smartphone-based system called Procedural Autonomy and Supervision System (PASS), which uses the Zwisch autonomy scale to facilitate assessment of the operative performances of surgical residents and promote progressive autonomy. To determine whether the use of PASS in a general surgery residency program is associated with any negative consequences, we tested the null hypothesis that PASS implementation at our institution would not negatively affect resident or faculty satisfaction in the operating room (OR) nor increase mean OR times for cases performed together by residents and faculty. Methods: Mean OR times were obtained from the electronic medical record at Northwestern Memorial Hospital for the 20 procedures most commonly performed by faculty members with residents before and after PASS implementation. OR times were compared via two-sample t-test. The OR Educational Environment Measure tool was used to assess OR satisfaction with all clinically active general surgery residents (n = 31) and full-time general surgery faculty members (n = 27) before and after PASS implementation. Results were compared using the Mann-Whitney rank sum test. Results: A significant prolongation in mean OR time between control and study period was found for only 1 of the 20 operative procedures performed at least 20 times by participating faculty members with residents. Based on the overall survey score, no significant differences were found between resident and faculty responses to the OR Educational Environment Measure survey before and after PASS implementation. When individual survey items were compared, while no differences were found with resident responses, differences were noted with faculty responses for 7 of the 35 items addressed although after Bonferroni correction none of these differences remained significant. Conclusions: Our data suggest that PASS does not increase mean OR times for the most commonly performed procedures. Resident OR satisfaction did not significantly change during PASS implementation, whereas some changes in faculty satisfaction were noted suggesting that PASS implementation may have had some negative effect with them. Although the effect on faculty satisfaction clearly requires further investigation, our findings support that use of an autonomy-based OR performance assessment system such as PASS does not appear to have a major negative influence on OR times nor OR satisfaction.",
keywords = "autonomy, evaluation, graduate medical education, surgery",
author = "Fryer, {Jonathan Paul} and Teitelbaum, {Ezra Nathaniel} and George, {Brian C.} and Schuller, {Mary C.} and Meyerson, {Shari Lynn} and Theodorou, {Christina M.} and Joseph Kang and Amy Yang and Lihui Zhao and Debra DaRosa",
year = "2018",
month = "3",
day = "1",
doi = "10.1016/j.jsurg.2016.11.018",
language = "English (US)",
volume = "75",
pages = "333--343",
journal = "Journal of Surgical Education",
issn = "1931-7204",
publisher = "Elsevier Inc.",
number = "2",

}

Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment. / Fryer, Jonathan Paul; Teitelbaum, Ezra Nathaniel; George, Brian C.; Schuller, Mary C.; Meyerson, Shari Lynn; Theodorou, Christina M.; Kang, Joseph; Yang, Amy; Zhao, Lihui; DaRosa, Debra.

In: Journal of Surgical Education, Vol. 75, No. 2, 01.03.2018, p. 333-343.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Effect of Ongoing Assessment of Resident Operative Autonomy on the Operating Room Environment

AU - Fryer, Jonathan Paul

AU - Teitelbaum, Ezra Nathaniel

AU - George, Brian C.

AU - Schuller, Mary C.

AU - Meyerson, Shari Lynn

AU - Theodorou, Christina M.

AU - Kang, Joseph

AU - Yang, Amy

AU - Zhao, Lihui

AU - DaRosa, Debra

PY - 2018/3/1

Y1 - 2018/3/1

N2 - Objective: We have previously demonstrated the feasibility and validity of a smartphone-based system called Procedural Autonomy and Supervision System (PASS), which uses the Zwisch autonomy scale to facilitate assessment of the operative performances of surgical residents and promote progressive autonomy. To determine whether the use of PASS in a general surgery residency program is associated with any negative consequences, we tested the null hypothesis that PASS implementation at our institution would not negatively affect resident or faculty satisfaction in the operating room (OR) nor increase mean OR times for cases performed together by residents and faculty. Methods: Mean OR times were obtained from the electronic medical record at Northwestern Memorial Hospital for the 20 procedures most commonly performed by faculty members with residents before and after PASS implementation. OR times were compared via two-sample t-test. The OR Educational Environment Measure tool was used to assess OR satisfaction with all clinically active general surgery residents (n = 31) and full-time general surgery faculty members (n = 27) before and after PASS implementation. Results were compared using the Mann-Whitney rank sum test. Results: A significant prolongation in mean OR time between control and study period was found for only 1 of the 20 operative procedures performed at least 20 times by participating faculty members with residents. Based on the overall survey score, no significant differences were found between resident and faculty responses to the OR Educational Environment Measure survey before and after PASS implementation. When individual survey items were compared, while no differences were found with resident responses, differences were noted with faculty responses for 7 of the 35 items addressed although after Bonferroni correction none of these differences remained significant. Conclusions: Our data suggest that PASS does not increase mean OR times for the most commonly performed procedures. Resident OR satisfaction did not significantly change during PASS implementation, whereas some changes in faculty satisfaction were noted suggesting that PASS implementation may have had some negative effect with them. Although the effect on faculty satisfaction clearly requires further investigation, our findings support that use of an autonomy-based OR performance assessment system such as PASS does not appear to have a major negative influence on OR times nor OR satisfaction.

AB - Objective: We have previously demonstrated the feasibility and validity of a smartphone-based system called Procedural Autonomy and Supervision System (PASS), which uses the Zwisch autonomy scale to facilitate assessment of the operative performances of surgical residents and promote progressive autonomy. To determine whether the use of PASS in a general surgery residency program is associated with any negative consequences, we tested the null hypothesis that PASS implementation at our institution would not negatively affect resident or faculty satisfaction in the operating room (OR) nor increase mean OR times for cases performed together by residents and faculty. Methods: Mean OR times were obtained from the electronic medical record at Northwestern Memorial Hospital for the 20 procedures most commonly performed by faculty members with residents before and after PASS implementation. OR times were compared via two-sample t-test. The OR Educational Environment Measure tool was used to assess OR satisfaction with all clinically active general surgery residents (n = 31) and full-time general surgery faculty members (n = 27) before and after PASS implementation. Results were compared using the Mann-Whitney rank sum test. Results: A significant prolongation in mean OR time between control and study period was found for only 1 of the 20 operative procedures performed at least 20 times by participating faculty members with residents. Based on the overall survey score, no significant differences were found between resident and faculty responses to the OR Educational Environment Measure survey before and after PASS implementation. When individual survey items were compared, while no differences were found with resident responses, differences were noted with faculty responses for 7 of the 35 items addressed although after Bonferroni correction none of these differences remained significant. Conclusions: Our data suggest that PASS does not increase mean OR times for the most commonly performed procedures. Resident OR satisfaction did not significantly change during PASS implementation, whereas some changes in faculty satisfaction were noted suggesting that PASS implementation may have had some negative effect with them. Although the effect on faculty satisfaction clearly requires further investigation, our findings support that use of an autonomy-based OR performance assessment system such as PASS does not appear to have a major negative influence on OR times nor OR satisfaction.

KW - autonomy

KW - evaluation

KW - graduate medical education

KW - surgery

UR - http://www.scopus.com/inward/record.url?scp=85016770162&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85016770162&partnerID=8YFLogxK

U2 - 10.1016/j.jsurg.2016.11.018

DO - 10.1016/j.jsurg.2016.11.018

M3 - Article

VL - 75

SP - 333

EP - 343

JO - Journal of Surgical Education

JF - Journal of Surgical Education

SN - 1931-7204

IS - 2

ER -