Observational Teamwork Assessment for Surgery: Content Validation and Tool Refinement

TY - JOUR

T1 - Observational Teamwork Assessment for Surgery:

T2 - Content Validation and Tool Refinement

AU - Hull, Louise

AU - Arora, Sonal

AU - Kassab, Eva

AU - Kneebone, Roger

AU - Sevdalis, Nick

N1 - Copyright © 2010 American College of Surgeons. Published by Elsevier Inc. All rights reserved.

PY - 2011/2

Y1 - 2011/2

N2 - BACKGROUND: Effective teamwork is crucial for safe surgery. Failures in nontechnical and teamwork skills are frequently implicated in adverse events. The Observational Teamwork Assessment for Surgery (OTAS) tool assesses teamwork of the entire team in the operating room. Empirical testing of OTAS has yet to explore the content validity of the tool.STUDY DESIGN: This was a cross-sectional observational study. Data were collected in 30 procedures by 2 trained researchers. Five teamwork behaviors were scored (ie, communication, leadership, cooperation, coordination, and monitoring) and behavior exemplar completion was recorded (phase 1). Expert operating room personnel (5 surgeons, 5 anesthesiologists, and 5 scrub nurses) assessed the content validity of the OTAS exemplar behaviors. Finally, a panel of operating room patient-safety experts refined the exemplars (phase 2).RESULTS: In total, the observability (presence/absence) of 130 exemplars was assessed by 2 blinded observers in 30 general surgical cases. Observer agreement was high (Cohen's κ ≥ 0.41) for 83.85% (109 of 130) of exemplar behaviors; 60.77% (79 of 130) of exemplar behaviors were observed frequently with high observer agreement. The majority of the exemplars were rated by expert operating room practitioners and an expert panel as substantial contributors to teamwork and patient safety. Based on expert consensus, 21 behavior exemplars were removed from OTAS and an additional 23 were modified.CONCLUSIONS: The exemplars of OTAS demonstrated very good content validity. Taken together with recent evidence on the construct validity of the tool, these findings demonstrate that OTAS is psychometrically robust for capturing teamwork in the operating room.

AB - BACKGROUND: Effective teamwork is crucial for safe surgery. Failures in nontechnical and teamwork skills are frequently implicated in adverse events. The Observational Teamwork Assessment for Surgery (OTAS) tool assesses teamwork of the entire team in the operating room. Empirical testing of OTAS has yet to explore the content validity of the tool.STUDY DESIGN: This was a cross-sectional observational study. Data were collected in 30 procedures by 2 trained researchers. Five teamwork behaviors were scored (ie, communication, leadership, cooperation, coordination, and monitoring) and behavior exemplar completion was recorded (phase 1). Expert operating room personnel (5 surgeons, 5 anesthesiologists, and 5 scrub nurses) assessed the content validity of the OTAS exemplar behaviors. Finally, a panel of operating room patient-safety experts refined the exemplars (phase 2).RESULTS: In total, the observability (presence/absence) of 130 exemplars was assessed by 2 blinded observers in 30 general surgical cases. Observer agreement was high (Cohen's κ ≥ 0.41) for 83.85% (109 of 130) of exemplar behaviors; 60.77% (79 of 130) of exemplar behaviors were observed frequently with high observer agreement. The majority of the exemplars were rated by expert operating room practitioners and an expert panel as substantial contributors to teamwork and patient safety. Based on expert consensus, 21 behavior exemplars were removed from OTAS and an additional 23 were modified.CONCLUSIONS: The exemplars of OTAS demonstrated very good content validity. Taken together with recent evidence on the construct validity of the tool, these findings demonstrate that OTAS is psychometrically robust for capturing teamwork in the operating room.

KW - Clinical Competence

KW - Communication

KW - Cooperative Behavior

KW - Cross-Sectional Studies

KW - Health Personnel

KW - Humans

KW - Interprofessional Relations

KW - Leadership

KW - Observer Variation

KW - Operating Rooms

KW - Patient Care Team

KW - Physicians

KW - Reproducibility of Results

KW - Surgical Procedures, Operative

U2 - 10.1016/j.jamcollsurg.2010.11.001

DO - 10.1016/j.jamcollsurg.2010.11.001

M3 - Article

C2 - 21276535

SN - 1072-7515

VL - 212

SP - 234

EP - 243

JO - Journal of the American College of Surgeons

JF - Journal of the American College of Surgeons

IS - 2

ER -