Machine-learning algorithm in acute stroke: real-world experience

N Chan; N Sibtain; T Booth; P de Souza; S Bibby; Y-H Mah; J Teo; J M U-King-Im

doi:10.1016/j.crad.2022.10.007

Machine-learning algorithm in acute stroke: real-world experience

N Chan, N Sibtain, T Booth, P de Souza, S Bibby, Y-H Mah, J Teo, J M U-King-Im

Research output: Contribution to journal › Article › peer-review

6 Citations (Scopus)

141 Downloads (Pure)

Abstract

AIM: To assess the clinical performance of a commercially available machine learning (ML) algorithm in acute stroke.

MATERIALS AND METHODS: CT and CT angiography (CTA) studies of 104 consecutive patients (43 females, age range 19-93, median age 62) performed for suspected acute stroke at a single tertiary institution with real-time ML software analysis (RAPID™ ASPECTS and CTA) were included. Studies were retrospectively reviewed independently by two neuroradiologists in a blinded manner.

RESULTS: The cohort included 24 acute infarcts and 16 large vessel occlusions (LVO). RAPID™ ASPECTS interpretation demonstrated high sensitivity (87.5%) and NPV (87.5%) but very poor specificity (30.9%) and PPV (30.9%) for detection of acute ischaemic parenchymal changes. There was a high percentage of false positives (51.1%). In cases of proven LVO, RAPID™ ASPECTS showed good correlation with neuroradiologists' blinded independent interpretation, Pearson correlation coefficient = 0.96 (both readers), 0.63 (RAPID™ vs reader 1), 0.69 (RAPID™ vs reader 2). RAPID™ CTA interpretation demonstrated high sensitivity (92.3%), specificity (85.3%), and negative predictive (NPV) (98.5%) with moderate positive predictive value (PPV) (52.2%) for detection of LVO (N=13). False positives accounted for 12.5% of cases, of which 27.3% were attributed to arterial stenosis.

CONCLUSION: RAPID™ CTA was robust and reliable in detection of LVO. Although demonstrating high sensitivity and NPV, RAPID™ ASPECTS interpretation was associated with a high number of false positives, which decreased clinicians' confidence in the algorithm. However, in cases of proven LVO, RAPID™ ASPECTS performed well and had good correlation with neuroradiologists' blinded interpretation.

Original language	English
Journal	Clinical Radiology
Volume	22
Early online date	18 Nov 2022
DOIs	https://doi.org/10.1016/j.crad.2022.10.007
Publication status	E-pub ahead of print - 18 Nov 2022

Access to Document

10.1016/j.crad.2022.10.007

Machine-learning algorithm in_CHAN_Publishedonline18November2022_GREEN AAMAccepted author manuscript, 2.31 MBLicence: CC BY-NC-ND

Cite this

@article{5f7b53d1cd3545359ae565968656b24d,

title = "Machine-learning algorithm in acute stroke: real-world experience",

abstract = "AIM: To assess the clinical performance of a commercially available machine learning (ML) algorithm in acute stroke.MATERIALS AND METHODS: CT and CT angiography (CTA) studies of 104 consecutive patients (43 females, age range 19-93, median age 62) performed for suspected acute stroke at a single tertiary institution with real-time ML software analysis (RAPID{\texttrademark} ASPECTS and CTA) were included. Studies were retrospectively reviewed independently by two neuroradiologists in a blinded manner.RESULTS: The cohort included 24 acute infarcts and 16 large vessel occlusions (LVO). RAPID{\texttrademark} ASPECTS interpretation demonstrated high sensitivity (87.5%) and NPV (87.5%) but very poor specificity (30.9%) and PPV (30.9%) for detection of acute ischaemic parenchymal changes. There was a high percentage of false positives (51.1%). In cases of proven LVO, RAPID{\texttrademark} ASPECTS showed good correlation with neuroradiologists' blinded independent interpretation, Pearson correlation coefficient = 0.96 (both readers), 0.63 (RAPID{\texttrademark} vs reader 1), 0.69 (RAPID{\texttrademark} vs reader 2). RAPID{\texttrademark} CTA interpretation demonstrated high sensitivity (92.3%), specificity (85.3%), and negative predictive (NPV) (98.5%) with moderate positive predictive value (PPV) (52.2%) for detection of LVO (N=13). False positives accounted for 12.5% of cases, of which 27.3% were attributed to arterial stenosis.CONCLUSION: RAPID{\texttrademark} CTA was robust and reliable in detection of LVO. Although demonstrating high sensitivity and NPV, RAPID{\texttrademark} ASPECTS interpretation was associated with a high number of false positives, which decreased clinicians' confidence in the algorithm. However, in cases of proven LVO, RAPID{\texttrademark} ASPECTS performed well and had good correlation with neuroradiologists' blinded interpretation.",

author = "N Chan and N Sibtain and T Booth and {de Souza}, P and S Bibby and Y-H Mah and J Teo and U-King-Im, {J M}",

note = "Funding Information: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Thomas C Booth reports financial support was provided by Wellcome Trust. James Teo declares that he has received research funding and support from InnovateUK, Health Data Research UK, Office of Life Sciences, NHSX, Nvidia, iRhythm Technologies, Bristol-Meyers-Squibb; has received speaker honorariums from Pfizer and Goldman Sachs; has received travel grant support from Bayer, has private practise at Cleveland Clinic London; and receives royalties from Wiley-Blackwells Publishing. Yee Mah reports a relationship with Medical Research Council that includes: funding grants. Publisher Copyright: {\textcopyright} 2022 The Royal College of Radiologists",

year = "2022",

month = nov,

day = "18",

doi = "10.1016/j.crad.2022.10.007",

language = "English",

volume = "22",

journal = "Clinical Radiology",

issn = "0009-9260",

publisher = "Elsevier",

}

TY - JOUR

T1 - Machine-learning algorithm in acute stroke

T2 - real-world experience

AU - Chan, N

AU - Sibtain, N

AU - Booth, T

AU - de Souza, P

AU - Bibby, S

AU - Mah, Y-H

AU - Teo, J

AU - U-King-Im, J M

N1 - Funding Information: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Thomas C Booth reports financial support was provided by Wellcome Trust. James Teo declares that he has received research funding and support from InnovateUK, Health Data Research UK, Office of Life Sciences, NHSX, Nvidia, iRhythm Technologies, Bristol-Meyers-Squibb; has received speaker honorariums from Pfizer and Goldman Sachs; has received travel grant support from Bayer, has private practise at Cleveland Clinic London; and receives royalties from Wiley-Blackwells Publishing. Yee Mah reports a relationship with Medical Research Council that includes: funding grants. Publisher Copyright: © 2022 The Royal College of Radiologists

PY - 2022/11/18

Y1 - 2022/11/18

N2 - AIM: To assess the clinical performance of a commercially available machine learning (ML) algorithm in acute stroke.MATERIALS AND METHODS: CT and CT angiography (CTA) studies of 104 consecutive patients (43 females, age range 19-93, median age 62) performed for suspected acute stroke at a single tertiary institution with real-time ML software analysis (RAPID™ ASPECTS and CTA) were included. Studies were retrospectively reviewed independently by two neuroradiologists in a blinded manner.RESULTS: The cohort included 24 acute infarcts and 16 large vessel occlusions (LVO). RAPID™ ASPECTS interpretation demonstrated high sensitivity (87.5%) and NPV (87.5%) but very poor specificity (30.9%) and PPV (30.9%) for detection of acute ischaemic parenchymal changes. There was a high percentage of false positives (51.1%). In cases of proven LVO, RAPID™ ASPECTS showed good correlation with neuroradiologists' blinded independent interpretation, Pearson correlation coefficient = 0.96 (both readers), 0.63 (RAPID™ vs reader 1), 0.69 (RAPID™ vs reader 2). RAPID™ CTA interpretation demonstrated high sensitivity (92.3%), specificity (85.3%), and negative predictive (NPV) (98.5%) with moderate positive predictive value (PPV) (52.2%) for detection of LVO (N=13). False positives accounted for 12.5% of cases, of which 27.3% were attributed to arterial stenosis.CONCLUSION: RAPID™ CTA was robust and reliable in detection of LVO. Although demonstrating high sensitivity and NPV, RAPID™ ASPECTS interpretation was associated with a high number of false positives, which decreased clinicians' confidence in the algorithm. However, in cases of proven LVO, RAPID™ ASPECTS performed well and had good correlation with neuroradiologists' blinded interpretation.

AB - AIM: To assess the clinical performance of a commercially available machine learning (ML) algorithm in acute stroke.MATERIALS AND METHODS: CT and CT angiography (CTA) studies of 104 consecutive patients (43 females, age range 19-93, median age 62) performed for suspected acute stroke at a single tertiary institution with real-time ML software analysis (RAPID™ ASPECTS and CTA) were included. Studies were retrospectively reviewed independently by two neuroradiologists in a blinded manner.RESULTS: The cohort included 24 acute infarcts and 16 large vessel occlusions (LVO). RAPID™ ASPECTS interpretation demonstrated high sensitivity (87.5%) and NPV (87.5%) but very poor specificity (30.9%) and PPV (30.9%) for detection of acute ischaemic parenchymal changes. There was a high percentage of false positives (51.1%). In cases of proven LVO, RAPID™ ASPECTS showed good correlation with neuroradiologists' blinded independent interpretation, Pearson correlation coefficient = 0.96 (both readers), 0.63 (RAPID™ vs reader 1), 0.69 (RAPID™ vs reader 2). RAPID™ CTA interpretation demonstrated high sensitivity (92.3%), specificity (85.3%), and negative predictive (NPV) (98.5%) with moderate positive predictive value (PPV) (52.2%) for detection of LVO (N=13). False positives accounted for 12.5% of cases, of which 27.3% were attributed to arterial stenosis.CONCLUSION: RAPID™ CTA was robust and reliable in detection of LVO. Although demonstrating high sensitivity and NPV, RAPID™ ASPECTS interpretation was associated with a high number of false positives, which decreased clinicians' confidence in the algorithm. However, in cases of proven LVO, RAPID™ ASPECTS performed well and had good correlation with neuroradiologists' blinded interpretation.

UR - http://www.scopus.com/inward/record.url?scp=85142398238&partnerID=8YFLogxK

U2 - 10.1016/j.crad.2022.10.007

DO - 10.1016/j.crad.2022.10.007

M3 - Article

C2 - 36411087

SN - 0009-9260

VL - 22

JO - Clinical Radiology

JF - Clinical Radiology

ER -

Machine-learning algorithm in acute stroke: real-world experience

Abstract

Access to Document

Other files and links

Fingerprint

Cite this