Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks

TY - CHAP

T1 - Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks

AU - Hu, Yipeng

AU - Gibson, Eli

AU - Lee, Li-lin

AU - Xie, Weidi

AU - Barratt, Dean C.

AU - Vercauteren, Tom

AU - Noble, J. Alison

PY - 2017/9/9

Y1 - 2017/9/9

N2 - Sonography synthesis has a wide range of applications, including medical procedure simulation, clinical training and multimodality image registration. In this paper, we propose a machine learning approach to simulate ultrasound images at given 3D spatial locations (relative to the patient anatomy), based on conditional generative adversarial networks (GANs). In particular, we introduce a novel neural network architecture that can sample anatomically accurate images conditionally on spatial position of the (real or mock) freehand ultrasound probe. To ensure an effective and efficient spatial information assimilation, the proposed spatially-conditioned GANs take calibrated pixel coordinates in global physical space as conditioning input, and utilise residual network units and shortcuts of conditioning data in the GANs’ discriminator and generator, respectively. Using optically tracked B-mode ultrasound images, acquired by an experienced sonographer on a fetus phantom, we demonstrate the feasibility of the proposed method by two sets of quantitative results: distances were calculated between corresponding anatomical landmarks identified in the held-out ultrasound images and the simulated data at the same locations unseen to the networks; a usability study was carried out to distinguish the simulated data from the real images. In summary, we present what we believe are state-of-the-art visually realistic ultrasound images, simulated by the proposed GAN architecture that is stable to train and capable of generating plausibly diverse image samples.

AB - Sonography synthesis has a wide range of applications, including medical procedure simulation, clinical training and multimodality image registration. In this paper, we propose a machine learning approach to simulate ultrasound images at given 3D spatial locations (relative to the patient anatomy), based on conditional generative adversarial networks (GANs). In particular, we introduce a novel neural network architecture that can sample anatomically accurate images conditionally on spatial position of the (real or mock) freehand ultrasound probe. To ensure an effective and efficient spatial information assimilation, the proposed spatially-conditioned GANs take calibrated pixel coordinates in global physical space as conditioning input, and utilise residual network units and shortcuts of conditioning data in the GANs’ discriminator and generator, respectively. Using optically tracked B-mode ultrasound images, acquired by an experienced sonographer on a fetus phantom, we demonstrate the feasibility of the proposed method by two sets of quantitative results: distances were calculated between corresponding anatomical landmarks identified in the held-out ultrasound images and the simulated data at the same locations unseen to the networks; a usability study was carried out to distinguish the simulated data from the real images. In summary, we present what we believe are state-of-the-art visually realistic ultrasound images, simulated by the proposed GAN architecture that is stable to train and capable of generating plausibly diverse image samples.

UR - http://www.scopus.com/inward/record.url?scp=85029582241&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-67564-0_11

DO - 10.1007/978-3-319-67564-0_11

M3 - Conference paper

AN - SCOPUS:85029582241

SN - 9783319675633

VL - 10555 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 105

EP - 115

BT - Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment - 5th International Workshop, CMMI 2017 2nd International Workshop, RAMBO 2017 and 1st International Workshop, SWITCH 2017 Held in Conjunction with MICCAI 2017, Proceedings

PB - Springer Verlag

T2 - 5th International Workshop on Computational Methods for Molecular Imaging, CMMI 2017, 2nd International Workshop on Reconstruction and Analysis of Moving Body Organs, RAMBO 2017 and 1st International Stroke Workshop on Imaging and Treatment Challenges, SWITCH 2017 held in Conjunction with 20th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2017

Y2 - 14 September 2017 through 14 September 2017

ER -