Abstract
Background and Aims
We aimed to develop, validate and compare statistical and machine learning (ML) models for predicting the risk of 30-day mortality after hospital admission for stroke.
Methods
Data from the UK Sentinel Stroke National Audit Program from 2013 to 2019 were used. XGBoost, Logistic Regression (LR), LR with elastic net, and LR with elastic net and interaction term models were developed using 80% randomly selected 2013 to 2018 admissions, internally validated on the 20% remaining admissions, and temporally validated on all 2019 admissions. The models were developed with 30 variables chosen from expert advice and literature review. An LR reference model was developed with 4 variables. Performance of the risk prediction models was evaluated in terms of discrimination, calibration, reclassification, Brier scores, and Decision-curve analysis.
Results
Data from 488, 497 patients were used, with an overall 30-day mortality of 12.3%. In the 2019 temporal validation set, XGBoost model obtained the lowest Brier score of 0.069 (95% CI: 0.068–0.071) and the highest AUC 0.895 (95% CI: 0.891–0.900). Adding more variables improved the accuracy of all models. The XGBoost model appropriately reclassified 1648 (8.1%) cases as being moderate or high risk which was deemed low risk by the LR reference model.
Conclusions
The potential gain for ML versus carefully developed statistical models to produce more accurate risk predictions of stroke mortality is likely to be modest. These findings emphasise the usefulness of collecting more detailed clinical data to support predictive analytics in stroke care.
We aimed to develop, validate and compare statistical and machine learning (ML) models for predicting the risk of 30-day mortality after hospital admission for stroke.
Methods
Data from the UK Sentinel Stroke National Audit Program from 2013 to 2019 were used. XGBoost, Logistic Regression (LR), LR with elastic net, and LR with elastic net and interaction term models were developed using 80% randomly selected 2013 to 2018 admissions, internally validated on the 20% remaining admissions, and temporally validated on all 2019 admissions. The models were developed with 30 variables chosen from expert advice and literature review. An LR reference model was developed with 4 variables. Performance of the risk prediction models was evaluated in terms of discrimination, calibration, reclassification, Brier scores, and Decision-curve analysis.
Results
Data from 488, 497 patients were used, with an overall 30-day mortality of 12.3%. In the 2019 temporal validation set, XGBoost model obtained the lowest Brier score of 0.069 (95% CI: 0.068–0.071) and the highest AUC 0.895 (95% CI: 0.891–0.900). Adding more variables improved the accuracy of all models. The XGBoost model appropriately reclassified 1648 (8.1%) cases as being moderate or high risk which was deemed low risk by the LR reference model.
Conclusions
The potential gain for ML versus carefully developed statistical models to produce more accurate risk predictions of stroke mortality is likely to be modest. These findings emphasise the usefulness of collecting more detailed clinical data to support predictive analytics in stroke care.
| Original language | English |
|---|---|
| Article number | O0078/#1175 |
| Pages (from-to) | 53-54 |
| Number of pages | 2 |
| Journal | European Stroke Journal |
| Volume | 6 |
| Issue number | Issue 1_suppl |
| DOIs | |
| Publication status | Published - 1 Sept 2021 |
