B-line Detection and Localization in Lung Ultrasound Videos Using Spatiotemporal Attention

TY - JOUR

T1 - B-line Detection and Localization in Lung Ultrasound Videos Using Spatiotemporal Attention

AU - Kerdegari, Hamideh

AU - Nhat, Phung Tran Huy

AU - McBride, Angela

AU - Pisani, Luigi

AU - Van Hao, Nguyen

AU - Bich Thuy, Duong

AU - Razavi, Reza

AU - Thwaites, Louise

AU - Yacoub, Sophie

AU - Gomez Herrero, Alberto

N1 - Funding Information: Funding: This work was supported by the Wellcome Trust UK (110179/Z/15/Z, 203905/Z/16/Z). H. Kerdegari, P. Nhat, R. Razavi and A. Gomez also acknowledge financial support from the Department of Health via the National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s and St Thomas’ NHS Foundation Trust in partnership with King’s College London and King’s College Hospital NHS Foundation Trust. Publisher Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

PY - 2021/12/9

Y1 - 2021/12/9

N2 - The presence of B-line artefacts, the main artefact reflecting lung abnormalities in dengue patients, is often assessed using lung ultrasound (LUS) imaging. Inspired by human visual attention that enables us to process videos efficiently by paying attention to where and when it is required, we propose a spatiotemporal attention mechanism for B-line detection in LUS videos. The spatial attention allows the model to focus on the most task relevant parts of the image by learning a saliency map. The temporal attention generates an attention score for each attended frame to identify the most relevant frames from an input video. Our model not only identifies videos where B-lines show, but also localizes, within those videos, B-line related features both spatially and temporally, despite being trained in a weakly-supervised manner. We evaluate our approach on a LUS video dataset collected from severe dengue patients in a resource-limited hospital, assessing the B-line detection rate and the model’s ability to localize discriminative B-line regions spatially and B-line frames temporally. Experimental results demonstrate the efficacy of our approach for classifying B-line videos with an F1 score of up to 83.2% and localizing the most salient B-line regions both spatially and temporally with a correlation coefficient of 0.67 and an IoU of 69.7%, respectively.

AB - The presence of B-line artefacts, the main artefact reflecting lung abnormalities in dengue patients, is often assessed using lung ultrasound (LUS) imaging. Inspired by human visual attention that enables us to process videos efficiently by paying attention to where and when it is required, we propose a spatiotemporal attention mechanism for B-line detection in LUS videos. The spatial attention allows the model to focus on the most task relevant parts of the image by learning a saliency map. The temporal attention generates an attention score for each attended frame to identify the most relevant frames from an input video. Our model not only identifies videos where B-lines show, but also localizes, within those videos, B-line related features both spatially and temporally, despite being trained in a weakly-supervised manner. We evaluate our approach on a LUS video dataset collected from severe dengue patients in a resource-limited hospital, assessing the B-line detection rate and the model’s ability to localize discriminative B-line regions spatially and B-line frames temporally. Experimental results demonstrate the efficacy of our approach for classifying B-line videos with an F1 score of up to 83.2% and localizing the most salient B-line regions both spatially and temporally with a correlation coefficient of 0.67 and an IoU of 69.7%, respectively.

UR - http://www.scopus.com/inward/record.url?scp=85121004995&partnerID=8YFLogxK

U2 - 10.3390/app112411697

DO - 10.3390/app112411697

M3 - Article

SN - 2076-3417

VL - 11

JO - Applied Sciences (Switzerland)

JF - Applied Sciences (Switzerland)

IS - 24

M1 - 11697

ER -