TY - CHAP
T1 - Longitudinal detection of radiological abnormalities with time-modulated lstm
AU - Santeramo, Ruggiero
AU - Withey, Samuel
AU - Montana, Giovanni
PY - 2018/1/1
Y1 - 2018/1/1
N2 - Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.
AB - Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.
KW - CNN
KW - Deep learning
KW - LSTM
KW - Medical imaging
KW - Time-modulated LSTM
KW - X-rays
UR - http://www.scopus.com/inward/record.url?scp=85057272303&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-00889-5_37
DO - 10.1007/978-3-030-00889-5_37
M3 - Conference paper
AN - SCOPUS:85057272303
SN - 9783030008888
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 326
EP - 333
BT - Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018
A2 - Maier-Hein, Lena
A2 - Syeda-Mahmood, Tanveer
A2 - Taylor, Zeike
A2 - Lu, Zhi
A2 - Stoyanov, Danail
A2 - Madabhushi, Anant
A2 - Tavares, João Manuel R.S.
A2 - Nascimento, Jacinto C.
A2 - Moradi, Mehdi
A2 - Martel, Anne
A2 - Papa, Joao Paulo
A2 - Conjeti, Sailesh
A2 - Belagiannis, Vasileios
A2 - Greenspan, Hayit
A2 - Carneiro, Gustavo
A2 - Bradley, Andrew
PB - Springer Verlag
T2 - 4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018
Y2 - 20 September 2018 through 20 September 2018
ER -