King's College London

Research portal

Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system

Research output: Contribution to journalArticle

Standard

Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. / Raison, Nicholas; Wood, Thomas; Brunckhorst, Oliver; Abe, Takashige; Ross, Talisa; Challacombe, Ben; Khan, Mohammed Shamim; Novara, Giacomo; Buffi, Nicolo; Van Der Poel, Henk; McIlhenny, Craig; Dasgupta, Prokar; Ahmed, Kamran.

In: Surgical endoscopy, Vol. 31, No. 12, 12.2017, p. 5403-5410.

Research output: Contribution to journalArticle

Harvard

Raison, N, Wood, T, Brunckhorst, O, Abe, T, Ross, T, Challacombe, B, Khan, MS, Novara, G, Buffi, N, Van Der Poel, H, McIlhenny, C, Dasgupta, P & Ahmed, K 2017, 'Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system', Surgical endoscopy, vol. 31, no. 12, pp. 5403-5410. https://doi.org/10.1007/s00464-017-5622-x

APA

Raison, N., Wood, T., Brunckhorst, O., Abe, T., Ross, T., Challacombe, B., ... Ahmed, K. (2017). Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. Surgical endoscopy, 31(12), 5403-5410. https://doi.org/10.1007/s00464-017-5622-x

Vancouver

Raison N, Wood T, Brunckhorst O, Abe T, Ross T, Challacombe B et al. Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. Surgical endoscopy. 2017 Dec;31(12):5403-5410. https://doi.org/10.1007/s00464-017-5622-x

Author

Raison, Nicholas ; Wood, Thomas ; Brunckhorst, Oliver ; Abe, Takashige ; Ross, Talisa ; Challacombe, Ben ; Khan, Mohammed Shamim ; Novara, Giacomo ; Buffi, Nicolo ; Van Der Poel, Henk ; McIlhenny, Craig ; Dasgupta, Prokar ; Ahmed, Kamran. / Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. In: Surgical endoscopy. 2017 ; Vol. 31, No. 12. pp. 5403-5410.

Bibtex Download

@article{9dba54c53e1c4a58a0080fe31d055916,
title = "Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system",
abstract = "BACKGROUND: Non-technical skills (NTS) are being increasingly recognised as vital for safe surgical practice. Numerous NTS rating systems have been developed to support effective training and assessment. Yet despite the additional challenges posed by robotic surgery, no NTS rating systems have been developed for this unique surgical environment. This study reports the development and validation of the first NTS behavioural rating system for robotic surgery.METHODS: A comprehensive index of all relevant NTS behaviours in robotic surgery was developed through observation of robotic theatre and interviews with robotic surgeons. Using a Delphi methodology, a panel of 16 expert surgeons was consulted to identify behaviours important to NTS assessment. These behaviours were organised into an appropriate assessment template. Experts were consulted on the feasibility, applicability and educational impact of ICARS. An observational trial was used to validate ICARS. 73 novice, intermediate and expert robotic surgeons completed a urethrovesical anastomosis within a simulated operating room. NTS were tested using four scripted scenarios of increasing difficulty. Performances were video recorded. Robotic and NTS experts assessed the videos post hoc using ICARS and the standard behavioural rating system, NOn-Technical Skills for Surgeons (NOTSS).RESULTS: 28 key non-technical behaviours were identified by the expert panel. The finalised behavioural rating system was organised into four principle domains and seven categories. Expert opinion strongly supported its implementation. ICARS was found to be equivalent to NOTSS on Bland-Altman analysis and accurately differentiated between novice, intermediate and expert participants, p = 0.01. Moderate agreement was found between raters, Krippendorff's alpha = 0.4. The internal structure of ICARS was shown to be consistent and reliable (median Cronbach alpha = 0.92, range 0.85-0.94).CONCLUSION: ICARS is the first NTS behavioural rating system developed for robotic surgery. Initial validation has shown it to be an effective and reliable tool. Implementation of ICARS will supported structured training and assessment of NTS within the robotic surgical curriculum.",
keywords = "Clinical Competence/standards, Curriculum, Educational Measurement, Humans, Operating Rooms, Program Evaluation, Robotic Surgical Procedures/education, Surgeons/education, Video Recording",
author = "Nicholas Raison and Thomas Wood and Oliver Brunckhorst and Takashige Abe and Talisa Ross and Ben Challacombe and Khan, {Mohammed Shamim} and Giacomo Novara and Nicolo Buffi and {Van Der Poel}, Henk and Craig McIlhenny and Prokar Dasgupta and Kamran Ahmed",
year = "2017",
month = "12",
doi = "10.1007/s00464-017-5622-x",
language = "English",
volume = "31",
pages = "5403--5410",
journal = "Surgical endoscopy",
issn = "0930-2794",
number = "12",

}

RIS (suitable for import to EndNote) Download

TY - JOUR

T1 - Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system

AU - Raison, Nicholas

AU - Wood, Thomas

AU - Brunckhorst, Oliver

AU - Abe, Takashige

AU - Ross, Talisa

AU - Challacombe, Ben

AU - Khan, Mohammed Shamim

AU - Novara, Giacomo

AU - Buffi, Nicolo

AU - Van Der Poel, Henk

AU - McIlhenny, Craig

AU - Dasgupta, Prokar

AU - Ahmed, Kamran

PY - 2017/12

Y1 - 2017/12

N2 - BACKGROUND: Non-technical skills (NTS) are being increasingly recognised as vital for safe surgical practice. Numerous NTS rating systems have been developed to support effective training and assessment. Yet despite the additional challenges posed by robotic surgery, no NTS rating systems have been developed for this unique surgical environment. This study reports the development and validation of the first NTS behavioural rating system for robotic surgery.METHODS: A comprehensive index of all relevant NTS behaviours in robotic surgery was developed through observation of robotic theatre and interviews with robotic surgeons. Using a Delphi methodology, a panel of 16 expert surgeons was consulted to identify behaviours important to NTS assessment. These behaviours were organised into an appropriate assessment template. Experts were consulted on the feasibility, applicability and educational impact of ICARS. An observational trial was used to validate ICARS. 73 novice, intermediate and expert robotic surgeons completed a urethrovesical anastomosis within a simulated operating room. NTS were tested using four scripted scenarios of increasing difficulty. Performances were video recorded. Robotic and NTS experts assessed the videos post hoc using ICARS and the standard behavioural rating system, NOn-Technical Skills for Surgeons (NOTSS).RESULTS: 28 key non-technical behaviours were identified by the expert panel. The finalised behavioural rating system was organised into four principle domains and seven categories. Expert opinion strongly supported its implementation. ICARS was found to be equivalent to NOTSS on Bland-Altman analysis and accurately differentiated between novice, intermediate and expert participants, p = 0.01. Moderate agreement was found between raters, Krippendorff's alpha = 0.4. The internal structure of ICARS was shown to be consistent and reliable (median Cronbach alpha = 0.92, range 0.85-0.94).CONCLUSION: ICARS is the first NTS behavioural rating system developed for robotic surgery. Initial validation has shown it to be an effective and reliable tool. Implementation of ICARS will supported structured training and assessment of NTS within the robotic surgical curriculum.

AB - BACKGROUND: Non-technical skills (NTS) are being increasingly recognised as vital for safe surgical practice. Numerous NTS rating systems have been developed to support effective training and assessment. Yet despite the additional challenges posed by robotic surgery, no NTS rating systems have been developed for this unique surgical environment. This study reports the development and validation of the first NTS behavioural rating system for robotic surgery.METHODS: A comprehensive index of all relevant NTS behaviours in robotic surgery was developed through observation of robotic theatre and interviews with robotic surgeons. Using a Delphi methodology, a panel of 16 expert surgeons was consulted to identify behaviours important to NTS assessment. These behaviours were organised into an appropriate assessment template. Experts were consulted on the feasibility, applicability and educational impact of ICARS. An observational trial was used to validate ICARS. 73 novice, intermediate and expert robotic surgeons completed a urethrovesical anastomosis within a simulated operating room. NTS were tested using four scripted scenarios of increasing difficulty. Performances were video recorded. Robotic and NTS experts assessed the videos post hoc using ICARS and the standard behavioural rating system, NOn-Technical Skills for Surgeons (NOTSS).RESULTS: 28 key non-technical behaviours were identified by the expert panel. The finalised behavioural rating system was organised into four principle domains and seven categories. Expert opinion strongly supported its implementation. ICARS was found to be equivalent to NOTSS on Bland-Altman analysis and accurately differentiated between novice, intermediate and expert participants, p = 0.01. Moderate agreement was found between raters, Krippendorff's alpha = 0.4. The internal structure of ICARS was shown to be consistent and reliable (median Cronbach alpha = 0.92, range 0.85-0.94).CONCLUSION: ICARS is the first NTS behavioural rating system developed for robotic surgery. Initial validation has shown it to be an effective and reliable tool. Implementation of ICARS will supported structured training and assessment of NTS within the robotic surgical curriculum.

KW - Clinical Competence/standards

KW - Curriculum

KW - Educational Measurement

KW - Humans

KW - Operating Rooms

KW - Program Evaluation

KW - Robotic Surgical Procedures/education

KW - Surgeons/education

KW - Video Recording

U2 - 10.1007/s00464-017-5622-x

DO - 10.1007/s00464-017-5622-x

M3 - Article

C2 - 28634630

VL - 31

SP - 5403

EP - 5410

JO - Surgical endoscopy

JF - Surgical endoscopy

SN - 0930-2794

IS - 12

ER -

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454