King's College London

Research portal

Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy

Research output: Contribution to journalArticle

Standard

Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy. / Ravì, Daniele; Szczotka, Agnieszka Barbara; Pereira, Stephen P; Vercauteren, Tom.

In: Medical Image Analysis, Vol. 53, 01.04.2019, p. 123-131.

Research output: Contribution to journalArticle

Harvard

Ravì, D, Szczotka, AB, Pereira, SP & Vercauteren, T 2019, 'Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy', Medical Image Analysis, vol. 53, pp. 123-131. https://doi.org/10.1016/j.media.2019.01.011

APA

Ravì, D., Szczotka, A. B., Pereira, S. P., & Vercauteren, T. (2019). Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy. Medical Image Analysis, 53, 123-131. https://doi.org/10.1016/j.media.2019.01.011

Vancouver

Ravì D, Szczotka AB, Pereira SP, Vercauteren T. Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy. Medical Image Analysis. 2019 Apr 1;53:123-131. https://doi.org/10.1016/j.media.2019.01.011

Author

Ravì, Daniele ; Szczotka, Agnieszka Barbara ; Pereira, Stephen P ; Vercauteren, Tom. / Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy. In: Medical Image Analysis. 2019 ; Vol. 53. pp. 123-131.

Bibtex Download

@article{14ed7f00d207462c8ba8674905fbd64c,
title = "Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy",
abstract = "In recent years, endomicroscopy has become increasingly used for diagnostic purposes and interventional guidance. It can provide intraoperative aids for real-time tissue characterization and can help to perform visual investigations aimed for example to discover epithelial cancers. Due to physical constraints on the acquisition process, endomicroscopy images, still today have a low number of informative pixels which hampers their quality. Post-processing techniques, such as Super-Resolution (SR), are a potential solution to increase the quality of these images. SR techniques are often supervised, requiring aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to train a model. However, in our domain, the lack of HR images hinders the collection of such pairs and makes supervised training unsuitable. For this reason, we propose an unsupervised SR framework based on an adversarial deep neural network with a physically-inspired cycle consistency, designed to impose some acquisition properties on the super-resolved images. Our framework can exploit HR images, regardless of the domain where they are coming from, to transfer the quality of the HR images to the initial LR images. This property can be particularly useful in all situations where pairs of LR/HR are not available during the training. Our quantitative analysis, validated using a database of 238 endomicroscopy video sequences from 143 patients, shows the ability of the pipeline to produce convincing super-resolved images. A Mean Opinion Score (MOS) study also confirms this quantitative image quality assessment.",
keywords = "Adversarial training, Cycle consistency, Deep learning, Probe-based confocal laser endomicroscopy, Unsupervised super-resolution",
author = "Daniele Rav{\`i} and Szczotka, {Agnieszka Barbara} and Pereira, {Stephen P} and Tom Vercauteren",
year = "2019",
month = apr,
day = "1",
doi = "10.1016/j.media.2019.01.011",
language = "English",
volume = "53",
pages = "123--131",
journal = "Medical Image Analysis",
issn = "1361-8415",

}

RIS (suitable for import to EndNote) Download

TY - JOUR

T1 - Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy

AU - Ravì, Daniele

AU - Szczotka, Agnieszka Barbara

AU - Pereira, Stephen P

AU - Vercauteren, Tom

PY - 2019/4/1

Y1 - 2019/4/1

N2 - In recent years, endomicroscopy has become increasingly used for diagnostic purposes and interventional guidance. It can provide intraoperative aids for real-time tissue characterization and can help to perform visual investigations aimed for example to discover epithelial cancers. Due to physical constraints on the acquisition process, endomicroscopy images, still today have a low number of informative pixels which hampers their quality. Post-processing techniques, such as Super-Resolution (SR), are a potential solution to increase the quality of these images. SR techniques are often supervised, requiring aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to train a model. However, in our domain, the lack of HR images hinders the collection of such pairs and makes supervised training unsuitable. For this reason, we propose an unsupervised SR framework based on an adversarial deep neural network with a physically-inspired cycle consistency, designed to impose some acquisition properties on the super-resolved images. Our framework can exploit HR images, regardless of the domain where they are coming from, to transfer the quality of the HR images to the initial LR images. This property can be particularly useful in all situations where pairs of LR/HR are not available during the training. Our quantitative analysis, validated using a database of 238 endomicroscopy video sequences from 143 patients, shows the ability of the pipeline to produce convincing super-resolved images. A Mean Opinion Score (MOS) study also confirms this quantitative image quality assessment.

AB - In recent years, endomicroscopy has become increasingly used for diagnostic purposes and interventional guidance. It can provide intraoperative aids for real-time tissue characterization and can help to perform visual investigations aimed for example to discover epithelial cancers. Due to physical constraints on the acquisition process, endomicroscopy images, still today have a low number of informative pixels which hampers their quality. Post-processing techniques, such as Super-Resolution (SR), are a potential solution to increase the quality of these images. SR techniques are often supervised, requiring aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to train a model. However, in our domain, the lack of HR images hinders the collection of such pairs and makes supervised training unsuitable. For this reason, we propose an unsupervised SR framework based on an adversarial deep neural network with a physically-inspired cycle consistency, designed to impose some acquisition properties on the super-resolved images. Our framework can exploit HR images, regardless of the domain where they are coming from, to transfer the quality of the HR images to the initial LR images. This property can be particularly useful in all situations where pairs of LR/HR are not available during the training. Our quantitative analysis, validated using a database of 238 endomicroscopy video sequences from 143 patients, shows the ability of the pipeline to produce convincing super-resolved images. A Mean Opinion Score (MOS) study also confirms this quantitative image quality assessment.

KW - Adversarial training

KW - Cycle consistency

KW - Deep learning

KW - Probe-based confocal laser endomicroscopy

KW - Unsupervised super-resolution

UR - http://www.scopus.com/inward/record.url?scp=85061339968&partnerID=8YFLogxK

U2 - 10.1016/j.media.2019.01.011

DO - 10.1016/j.media.2019.01.011

M3 - Article

VL - 53

SP - 123

EP - 131

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

ER -

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454