TY - JOUR
T1 - Intraoperative Assessment of Non-Technical Skills for Surgeons (NOTSS) and Qualitative Description of their Effects on Intraoperative Performance
AU - Abahuje, Egide
AU - Johnson, Julie
AU - Halverson, Amy
AU - Stulberg, Jonah J.
N1 - Publisher Copyright:
© 2022 Association of Program Directors in Surgery
PY - 2022/9/1
Y1 - 2022/9/1
N2 - OBJECTIVE: The aims of this study were (1) to evaluate the feasibility of using the Non-Technical Skills for Surgeons (NOTSS) rating tool in assessing surgeons’ non-technical skills behaviors in live operations, and (2) to describe the effect of NOTSS on intraoperative performance. SETTING DESIGN: This study was conducted in an academic hospital in North America. Two observers independently conducted direct non-participant observations using the NOTSS rating tool to assess non-technical skills, and to document examples of effective or ineffective non-technical skills behaviors. Observers took field notes to document non-technical skill gaps that were not captured by the NOTSS rating tool, and situations or scenarios that presented challenges for accurate assessment. Interclass correlation estimates and 95% confidence intervals were calculated to assess the validity of the NOTSS rating tool. Deductive thematic qualitative data analysis was used for field notes and NOTSS behavior descriptions. PARTICIPANTS: Participants were general surgeons performing either minimally invasive (robotic assisted or laparoscopic surgery), or open procedures. RESULTS: We observed 18 surgeries, involving 6 surgeons, 11 residents and one fellow resulting in 37 hours of direct observations. The mean NOTSS score was 3.8 (SD 0.41) for situation awareness, 3.75 (SD 0.47) for decision-making, 3.71 (SD 0.39) for communication and teamwork, and 3.76 (SD 0.38) for leadership. The inter-rater reliability ranged between 0.65 and 0.80 for each NOTSS categories. The observers documented examples of effective non-technical skills behaviors and examples of behaviors that need improvement. Furthermore, we described the effect of each observed behavior on intraoperative performance. One challenge to NOTSS use in live surgery was that observers had to infer situation awareness, decision-making, and coping with pressure as these were not easily observed without attending surgeons articulating their underlying thought process. CONCLUSION: The use of the NOTSS tool in live surgery is a valid and practical tool to document observed behaviors and their effect on intraoperative performance in order to provide constructive feedback to surgeons. One notable limitation is that without specific articulation by the surgeon of their underlying thought process the observer must infer specific elements. By documenting specific real-world events with high inter-rater reliability and adequate surgeon score variation the process can be used to provide useful feedback for improvement.
AB - OBJECTIVE: The aims of this study were (1) to evaluate the feasibility of using the Non-Technical Skills for Surgeons (NOTSS) rating tool in assessing surgeons’ non-technical skills behaviors in live operations, and (2) to describe the effect of NOTSS on intraoperative performance. SETTING DESIGN: This study was conducted in an academic hospital in North America. Two observers independently conducted direct non-participant observations using the NOTSS rating tool to assess non-technical skills, and to document examples of effective or ineffective non-technical skills behaviors. Observers took field notes to document non-technical skill gaps that were not captured by the NOTSS rating tool, and situations or scenarios that presented challenges for accurate assessment. Interclass correlation estimates and 95% confidence intervals were calculated to assess the validity of the NOTSS rating tool. Deductive thematic qualitative data analysis was used for field notes and NOTSS behavior descriptions. PARTICIPANTS: Participants were general surgeons performing either minimally invasive (robotic assisted or laparoscopic surgery), or open procedures. RESULTS: We observed 18 surgeries, involving 6 surgeons, 11 residents and one fellow resulting in 37 hours of direct observations. The mean NOTSS score was 3.8 (SD 0.41) for situation awareness, 3.75 (SD 0.47) for decision-making, 3.71 (SD 0.39) for communication and teamwork, and 3.76 (SD 0.38) for leadership. The inter-rater reliability ranged between 0.65 and 0.80 for each NOTSS categories. The observers documented examples of effective non-technical skills behaviors and examples of behaviors that need improvement. Furthermore, we described the effect of each observed behavior on intraoperative performance. One challenge to NOTSS use in live surgery was that observers had to infer situation awareness, decision-making, and coping with pressure as these were not easily observed without attending surgeons articulating their underlying thought process. CONCLUSION: The use of the NOTSS tool in live surgery is a valid and practical tool to document observed behaviors and their effect on intraoperative performance in order to provide constructive feedback to surgeons. One notable limitation is that without specific articulation by the surgeon of their underlying thought process the observer must infer specific elements. By documenting specific real-world events with high inter-rater reliability and adequate surgeon score variation the process can be used to provide useful feedback for improvement.
KW - crisis resource management
KW - formative feedback
KW - non-technical skills
KW - operating room
KW - patient safety
UR - http://www.scopus.com/inward/record.url?scp=85131120854&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85131120854&partnerID=8YFLogxK
U2 - 10.1016/j.jsurg.2022.04.010
DO - 10.1016/j.jsurg.2022.04.010
M3 - Article
C2 - 35637141
AN - SCOPUS:85131120854
SN - 1931-7204
VL - 79
SP - 1237
EP - 1245
JO - Journal of Surgical Education
JF - Journal of Surgical Education
IS - 5
ER -