Objective structured assessment of debriefing: bringing science to the art of debriefing in surgery

TY - JOUR

T1 - Objective structured assessment of debriefing

T2 - bringing science to the art of debriefing in surgery

AU - Arora, Sonal

AU - Ahmed, Maria

AU - Paige, John

AU - Nestel, Debra

AU - Runnacles, Jane

AU - Hull, Louise

AU - Darzi, Ara

AU - Sevdalis, Nick

PY - 2012/12

Y1 - 2012/12

N2 - OBJECTIVE: : To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings.INTRODUCTION: : Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing-yet there is a paucity of research to guide best practice.METHODS: : Phase 1-Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2-Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations.RESULTS: : Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test-retest ICC = 0.90], and face and content valid (content validity index = 0.94).CONCLUSIONS: : OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

AB - OBJECTIVE: : To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings.INTRODUCTION: : Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing-yet there is a paucity of research to guide best practice.METHODS: : Phase 1-Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2-Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations.RESULTS: : Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test-retest ICC = 0.90], and face and content valid (content validity index = 0.94).CONCLUSIONS: : OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

KW - Clinical Competence

KW - Educational Measurement

KW - Humans

KW - Specialties, Surgical

U2 - 10.1097/SLA.0b013e3182610c91

DO - 10.1097/SLA.0b013e3182610c91

M3 - Article

C2 - 22895396

SN - 0003-4932

VL - 256

SP - 982

EP - 988

JO - Annals of Surgery

JF - Annals of Surgery

IS - 6

ER -