Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models

Mark S. Graham; Walter Hugo Lopez Pinaya; Paul Wright; Petru-Daniel Tudosiu; Yee H. Mah; James T. Teo; H. Rolf Jäger; David Werring; Parashkev Nachev; Sebastien Ourselin; M. Jorge Cardoso

doi:10.1007/978-3-031-43907-0_43

Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models

Mark S. Graham^*, Walter Hugo Lopez Pinaya, Paul Wright, Petru-Daniel Tudosiu, Yee H. Mah, James T. Teo, H. Rolf Jäger, David Werring, Parashkev Nachev, Sebastien Ourselin, M. Jorge Cardoso

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

2 Citations (Scopus)

Abstract

Methods for out-of-distribution (OOD) detection that scale to 3D data are crucial components of any real-world clinical deep learning system. Classic denoising diffusion probabilistic models (DDPMs) have been recently proposed as a robust way to perform reconstruction-based OOD detection on 2D datasets, but do not trivially scale to 3D data. In this work, we propose to use Latent Diffusion Models (LDMs), which enable the scaling of DDPMs to high-resolution 3D medical data. We validate the proposed approach on near- and far-OOD datasets and compare it to a recently proposed, 3D-enabled approach using Latent Transformer Models (LTMs). Not only does the proposed LDM-based approach achieve statistically significant better performance, it also shows less sensitivity to the underlying latent representation, more favourable memory scaling, and produces better spatial anomaly maps. Code is available at https://github.com/marksgraham/ddpm-ood.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023
Subtitle of host publication	26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I
Editors	Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Place of Publication	Cham
Publisher	Springer Nature
Pages	446-456
Number of pages	11
ISBN (Electronic)	9783031439070
ISBN (Print)	9783031439063
DOIs	https://doi.org/10.1007/978-3-031-43907-0_43
Publication status	Published - 2 Oct 2023
Event	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher	Springer
Volume	14220
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/2023 → 12/10/2023

Keywords

Latent diffusion models
Out-of-distribution detection

Access to Document

10.1007/978-3-031-43907-0_43

Cite this

Graham, M. S., Pinaya, W. H. L., Wright, P., Tudosiu, P.-D., Mah, Y. H., Teo, J. T., Jäger, H. R., Werring, D., Nachev, P., Ourselin, S., & Cardoso, M. J. (2023). Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models. In H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I (pp. 446-456). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14220). Springer Nature. https://doi.org/10.1007/978-3-031-43907-0_43

Graham, Mark S. ; Pinaya, Walter Hugo Lopez ; Wright, Paul et al. / Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I. editor / Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Cham : Springer Nature, 2023. pp. 446-456 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{92287c0411d6401e960d2b76ad2513b4,

title = "Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models",

abstract = "Methods for out-of-distribution (OOD) detection that scale to 3D data are crucial components of any real-world clinical deep learning system. Classic denoising diffusion probabilistic models (DDPMs) have been recently proposed as a robust way to perform reconstruction-based OOD detection on 2D datasets, but do not trivially scale to 3D data. In this work, we propose to use Latent Diffusion Models (LDMs), which enable the scaling of DDPMs to high-resolution 3D medical data. We validate the proposed approach on near- and far-OOD datasets and compare it to a recently proposed, 3D-enabled approach using Latent Transformer Models (LTMs). Not only does the proposed LDM-based approach achieve statistically significant better performance, it also shows less sensitivity to the underlying latent representation, more favourable memory scaling, and produces better spatial anomaly maps. Code is available at https://github.com/marksgraham/ddpm-ood.",

keywords = "Latent diffusion models, Out-of-distribution detection",

author = "Graham, {Mark S.} and Pinaya, {Walter Hugo Lopez} and Paul Wright and Petru-Daniel Tudosiu and Mah, {Yee H.} and Teo, {James T.} and J{\"a}ger, {H. Rolf} and David Werring and Parashkev Nachev and Sebastien Ourselin and Cardoso, {M. Jorge}",

note = "Funding Information: Acknowledgements. MSG, WHLP, RG, PW, PN, SO, and MJC are supported by the Wellcome Trust (WT213038/Z/18/Z). MJC and SO are also supported by the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z), and the InnovateUK-funded London AI centre for Value-based Healthcare. PTD is supported by the EPSRC (EP/R513064/1). YM is supported by an MRC Clinical Academic Research Partnership grant (MR/T005351/1). PN is also supported by the UCLH NIHR Biomedical Research Centre. Datasets CROMIS and KCH were used with ethics 20/ES/0005. Funding Information: MSG, WHLP, RG, PW, PN, SO, and MJC are supported by the Wellcome Trust (WT213038/Z/18/Z). MJC and SO are also supported by the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z), and the InnovateUK-funded London AI centre for Value-based Healthcare. PTD is supported by the EPSRC (EP/R513064/1). YM is supported by an MRC Clinical Academic Research Partnership grant (MR/T005351/1). PN is also supported by the UCLH NIHR Biomedical Research Centre. Datasets CROMIS and KCH were used with ethics 20/ES/0005. Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

month = oct,

day = "2",

doi = "10.1007/978-3-031-43907-0_43",

language = "English",

isbn = "9783031439063",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Nature",

pages = "446--456",

editor = "Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023",

}

Graham, MS , Pinaya, WHL , Wright, P, Tudosiu, P-D, Mah, YH , Teo, JT, Jäger, HR, Werring, D, Nachev, P, Ourselin, S & Cardoso, MJ 2023, Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models. in H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14220, Springer Nature, Cham, pp. 446-456, 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, Canada, 8/10/2023. https://doi.org/10.1007/978-3-031-43907-0_43

Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models. / Graham, Mark S.; Pinaya, Walter Hugo Lopez ; Wright, Paul et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I. ed. / Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Cham: Springer Nature, 2023. p. 446-456 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14220).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › peer-review

TY - CHAP

T1 - Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models

AU - Graham, Mark S.

AU - Pinaya, Walter Hugo Lopez

AU - Wright, Paul

AU - Tudosiu, Petru-Daniel

AU - Mah, Yee H.

AU - Teo, James T.

AU - Jäger, H. Rolf

AU - Werring, David

AU - Nachev, Parashkev

AU - Ourselin, Sebastien

AU - Cardoso, M. Jorge

N1 - Funding Information: Acknowledgements. MSG, WHLP, RG, PW, PN, SO, and MJC are supported by the Wellcome Trust (WT213038/Z/18/Z). MJC and SO are also supported by the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z), and the InnovateUK-funded London AI centre for Value-based Healthcare. PTD is supported by the EPSRC (EP/R513064/1). YM is supported by an MRC Clinical Academic Research Partnership grant (MR/T005351/1). PN is also supported by the UCLH NIHR Biomedical Research Centre. Datasets CROMIS and KCH were used with ethics 20/ES/0005. Funding Information: MSG, WHLP, RG, PW, PN, SO, and MJC are supported by the Wellcome Trust (WT213038/Z/18/Z). MJC and SO are also supported by the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z), and the InnovateUK-funded London AI centre for Value-based Healthcare. PTD is supported by the EPSRC (EP/R513064/1). YM is supported by an MRC Clinical Academic Research Partnership grant (MR/T005351/1). PN is also supported by the UCLH NIHR Biomedical Research Centre. Datasets CROMIS and KCH were used with ethics 20/ES/0005. Publisher Copyright: © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2023/10/2

Y1 - 2023/10/2

N2 - Methods for out-of-distribution (OOD) detection that scale to 3D data are crucial components of any real-world clinical deep learning system. Classic denoising diffusion probabilistic models (DDPMs) have been recently proposed as a robust way to perform reconstruction-based OOD detection on 2D datasets, but do not trivially scale to 3D data. In this work, we propose to use Latent Diffusion Models (LDMs), which enable the scaling of DDPMs to high-resolution 3D medical data. We validate the proposed approach on near- and far-OOD datasets and compare it to a recently proposed, 3D-enabled approach using Latent Transformer Models (LTMs). Not only does the proposed LDM-based approach achieve statistically significant better performance, it also shows less sensitivity to the underlying latent representation, more favourable memory scaling, and produces better spatial anomaly maps. Code is available at https://github.com/marksgraham/ddpm-ood.

AB - Methods for out-of-distribution (OOD) detection that scale to 3D data are crucial components of any real-world clinical deep learning system. Classic denoising diffusion probabilistic models (DDPMs) have been recently proposed as a robust way to perform reconstruction-based OOD detection on 2D datasets, but do not trivially scale to 3D data. In this work, we propose to use Latent Diffusion Models (LDMs), which enable the scaling of DDPMs to high-resolution 3D medical data. We validate the proposed approach on near- and far-OOD datasets and compare it to a recently proposed, 3D-enabled approach using Latent Transformer Models (LTMs). Not only does the proposed LDM-based approach achieve statistically significant better performance, it also shows less sensitivity to the underlying latent representation, more favourable memory scaling, and produces better spatial anomaly maps. Code is available at https://github.com/marksgraham/ddpm-ood.

KW - Latent diffusion models

KW - Out-of-distribution detection

UR - http://www.scopus.com/inward/record.url?scp=85174593067&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43907-0_43

DO - 10.1007/978-3-031-43907-0_43

M3 - Chapter

AN - SCOPUS:85174593067

SN - 9783031439063

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 446

EP - 456

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer Nature

CY - Cham

T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

Graham MS , Pinaya WHL , Wright P, Tudosiu PD, Mah YH , Teo JT et al. Unsupervised 3D Out-of-Distribution Detection with Latent Diffusion Models. In Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part I. Cham: Springer Nature. 2023. p. 446-456. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2023 Oct 1. doi: 10.1007/978-3-031-43907-0_43