Longitudinal detection of radiological abnormalities with time-modulated lstm

Ruggiero Santeramo, Samuel Withey, Giovanni Montana*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

23 Citations (Scopus)

Abstract

Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.

Original languageEnglish
Title of host publicationDeep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018
EditorsLena Maier-Hein, Tanveer Syeda-Mahmood, Zeike Taylor, Zhi Lu, Danail Stoyanov, Anant Madabhushi, João Manuel R.S. Tavares, Jacinto C. Nascimento, Mehdi Moradi, Anne Martel, Joao Paulo Papa, Sailesh Conjeti, Vasileios Belagiannis, Hayit Greenspan, Gustavo Carneiro, Andrew Bradley
PublisherSpringer Verlag
Pages326-333
Number of pages8
ISBN (Print)9783030008888
DOIs
Publication statusPublished - 1 Jan 2018
Event4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018 - Granada, Spain
Duration: 20 Sept 201820 Sept 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11045 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018
Country/TerritorySpain
CityGranada
Period20/09/201820/09/2018

Keywords

  • CNN
  • Deep learning
  • LSTM
  • Medical imaging
  • Time-modulated LSTM
  • X-rays

Cite this