Factors affecting the labelling accuracy of brain MRI studies relevant for deep learning abnormality detection

Matthew Benger, David a. Wood, Sina Kafiabadi, Aisha Al busaidi, Emily Guilhem, Jeremy Lynch, Matthew Townend, Antanas Montvila, Juveria Siddiqui, Naveen Gadapa, Gareth Barker, Sebastian Ourselin, James H. Cole, Thomas C. Booth

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Unlocking the vast potential of deep learning-based computer vision classification systems necessitates large data sets for model training. Natural Language Processing (NLP)—involving automation of dataset labelling—represents a potential avenue to achieve this. However, many aspects of NLP for dataset labelling remain unvalidated. Expert radiologists manually labelled over 5,000 MRI head reports in order to develop a deep learning-based neuroradiology NLP report classifier. Our results demonstrate that binary labels (normal vs. abnormal) showed high rates of accuracy, even when only two MRI sequences (T2-weighted and those based on diffusion weighted imaging) were employed as opposed to all sequences in an examination. Meanwhile, the accuracy of more specific labelling for multiple disease categories was variable and dependent on the category. Finally, resultant model performance was shown to be dependent on the expertise of the original labeller, with worse performance seen with non-expert vs. expert labellers.
Original languageEnglish
JournalFrontiers in Radiology
Volume3
Early online date27 Nov 2023
DOIs
Publication statusE-pub ahead of print - 27 Nov 2023

Fingerprint

Dive into the research topics of 'Factors affecting the labelling accuracy of brain MRI studies relevant for deep learning abnormality detection'. Together they form a unique fingerprint.

Cite this