Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative

James P. Howard; Catherine C. Stowell; Graham D. Cole; Kajaluxy Ananthan; Camelia D. Demetrescu; Keith Pearce; Ronak Rajani; Jobanpreet Sehmi; Kavitha Vimalesvaran; G. Sunthar Kanaganayagam; Eleanor McPhail; Arjun K. Ghosh; John B. Chambers; Amar P. Singh; Massoud Zolgharni; Bushra Rana; Darrel P. Francis; Matthew J. Shun-Shin

doi:10.1161/CIRCIMAGING.120.011951

Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative

James P. Howard, Catherine C. Stowell, Graham D. Cole, Kajaluxy Ananthan, Camelia D. Demetrescu, Keith Pearce, Ronak Rajani, Jobanpreet Sehmi, Kavitha Vimalesvaran, G. Sunthar Kanaganayagam, Eleanor McPhail, Arjun K. Ghosh, John B. Chambers, Amar P. Singh, Massoud Zolgharni, Bushra Rana, Darrel P. Francis^*, Matthew J. Shun-Shin

^*Corresponding author for this work

King's College London

Research output: Contribution to journal › Article › peer-review

25 Citations (Scopus)

Abstract

Background: requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques. Methods: The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus. Results: In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles. Conclusions: Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.

Original language	English
Pages (from-to)	405-415
Number of pages	11
Journal	Circulation: Cardiovascular Imaging
DOIs	https://doi.org/10.1161/CIRCIMAGING.120.011951
Publication status	Accepted/In press - 2021

Keywords

consensus
echocardiography
hospital
left ventricle
machine learning

Access to Document

10.1161/CIRCIMAGING.120.011951

Cite this

Howard, J. P., Stowell, C. C., Cole, G. D., Ananthan, K., Demetrescu, C. D., Pearce, K., Rajani, R., Sehmi, J., Vimalesvaran, K., Kanaganayagam, G. S., McPhail, E., Ghosh, A. K., Chambers, J. B., Singh, A. P., Zolgharni, M., Rana, B., Francis, D. P., & Shun-Shin, M. J. (Accepted/In press). Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative. Circulation: Cardiovascular Imaging, 405-415. https://doi.org/10.1161/CIRCIMAGING.120.011951

@article{d451018e52aa4c23bbc472c558ff1c89,

title = "Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative",

abstract = "Background: requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques. Methods: The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus. Results: In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles. Conclusions: Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.",

keywords = "consensus, echocardiography, hospital, left ventricle, machine learning",

author = "Howard, {James P.} and Stowell, {Catherine C.} and Cole, {Graham D.} and Kajaluxy Ananthan and Demetrescu, {Camelia D.} and Keith Pearce and Ronak Rajani and Jobanpreet Sehmi and Kavitha Vimalesvaran and Kanaganayagam, {G. Sunthar} and Eleanor McPhail and Ghosh, {Arjun K.} and Chambers, {John B.} and Singh, {Amar P.} and Massoud Zolgharni and Bushra Rana and Francis, {Darrel P.} and Shun-Shin, {Matthew J.}",

note = "Publisher Copyright: {\textcopyright} 2021 The Authors. Circulation: Cardiovascular Imaging is published on behalf of the American Heart Association, Inc. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.",

year = "2021",

doi = "10.1161/CIRCIMAGING.120.011951",

language = "English",

pages = "405--415",

journal = "Circulation: Cardiovascular Imaging",

issn = "1941-9651",

publisher = "Lippincott Williams and Wilkins",

}

Howard, JP, Stowell, CC, Cole, GD, Ananthan, K, Demetrescu, CD, Pearce, K, Rajani, R, Sehmi, J, Vimalesvaran, K, Kanaganayagam, GS, McPhail, E, Ghosh, AK, Chambers, JB, Singh, AP, Zolgharni, M, Rana, B, Francis, DP & Shun-Shin, MJ 2021, 'Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative', Circulation: Cardiovascular Imaging, pp. 405-415. https://doi.org/10.1161/CIRCIMAGING.120.011951

TY - JOUR

T1 - Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative

AU - Howard, James P.

AU - Stowell, Catherine C.

AU - Cole, Graham D.

AU - Ananthan, Kajaluxy

AU - Demetrescu, Camelia D.

AU - Pearce, Keith

AU - Rajani, Ronak

AU - Sehmi, Jobanpreet

AU - Vimalesvaran, Kavitha

AU - Kanaganayagam, G. Sunthar

AU - McPhail, Eleanor

AU - Ghosh, Arjun K.

AU - Chambers, John B.

AU - Singh, Amar P.

AU - Zolgharni, Massoud

AU - Rana, Bushra

AU - Francis, Darrel P.

AU - Shun-Shin, Matthew J.

N1 - Publisher Copyright: © 2021 The Authors. Circulation: Cardiovascular Imaging is published on behalf of the American Heart Association, Inc. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.

PY - 2021

Y1 - 2021

N2 - Background: requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques. Methods: The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus. Results: In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles. Conclusions: Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.

AB - Background: requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques. Methods: The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus. Results: In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles. Conclusions: Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.

KW - consensus

KW - echocardiography

KW - hospital

KW - left ventricle

KW - machine learning

UR - http://www.scopus.com/inward/record.url?scp=85106182137&partnerID=8YFLogxK

U2 - 10.1161/CIRCIMAGING.120.011951

DO - 10.1161/CIRCIMAGING.120.011951

M3 - Article

C2 - 33998247

AN - SCOPUS:85106182137

SN - 1941-9651

SP - 405

EP - 415

JO - Circulation: Cardiovascular Imaging

JF - Circulation: Cardiovascular Imaging

ER -

Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this