DeepCut: Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks

Martin Rajchl; Matthew C H Lee; Ozan Oktay; Konstantinos Kamnitsas; Jonathan Passerat-Palmbach; Wenjia Bai; Mellisa Damodaram; Mary A. Rutherford; Joseph V. Hajnal; Bernhard Kainz; Daniel Rueckert

doi:10.1109/TMI.2016.2621185

DeepCut: Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks

Martin Rajchl^*, Matthew C H Lee, Ozan Oktay, Konstantinos Kamnitsas, Jonathan Passerat-Palmbach, Wenjia Bai, Mellisa Damodaram, Mary A. Rutherford, Joseph V. Hajnal, Bernhard Kainz, Daniel Rueckert

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

307 Citations (Scopus)

Abstract

In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.

Original language	English
Article number	7739993
Pages (from-to)	674-683
Number of pages	10
Journal	IEEE Transactions on Medical Imaging
Volume	36
Issue number	2
DOIs	https://doi.org/10.1109/TMI.2016.2621185
Publication status	Published - 1 Feb 2017

Keywords

Bounding box
convolutional neural networks
DeepCut
image segmentation
machine learning
weak annotations

Access to Document

10.1109/TMI.2016.2621185

Cite this

Rajchl, M., Lee, M. C. H., Oktay, O., Kamnitsas, K., Passerat-Palmbach, J., Bai, W., Damodaram, M., Rutherford, M. A., Hajnal, J. V., Kainz, B., & Rueckert, D. (2017). DeepCut: Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks. IEEE Transactions on Medical Imaging, 36(2), 674-683. Article 7739993. https://doi.org/10.1109/TMI.2016.2621185

@article{2b8f91451a8e41ce8df6bcce9c6a45d1,

title = "DeepCut: Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks",

abstract = "In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a na{\"i}ve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.",

keywords = "Bounding box, convolutional neural networks, DeepCut, image segmentation, machine learning, weak annotations",

author = "Martin Rajchl and Lee, {Matthew C H} and Ozan Oktay and Konstantinos Kamnitsas and Jonathan Passerat-Palmbach and Wenjia Bai and Mellisa Damodaram and Rutherford, {Mary A.} and Hajnal, {Joseph V.} and Bernhard Kainz and Daniel Rueckert",

year = "2017",

month = feb,

day = "1",

doi = "10.1109/TMI.2016.2621185",

language = "English",

volume = "36",

pages = "674--683",

journal = "IEEE Transactions on Medical Imaging",

issn = "0278-0062",

publisher = "I E E E",

number = "2",

}

TY - JOUR

T1 - DeepCut

T2 - Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks

AU - Rajchl, Martin

AU - Lee, Matthew C H

AU - Oktay, Ozan

AU - Kamnitsas, Konstantinos

AU - Passerat-Palmbach, Jonathan

AU - Bai, Wenjia

AU - Damodaram, Mellisa

AU - Rutherford, Mary A.

AU - Hajnal, Joseph V.

AU - Kainz, Bernhard

AU - Rueckert, Daniel

PY - 2017/2/1

Y1 - 2017/2/1

N2 - In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.

AB - In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.

KW - Bounding box

KW - convolutional neural networks

KW - DeepCut

KW - image segmentation

KW - machine learning

KW - weak annotations

UR - http://www.scopus.com/inward/record.url?scp=85012110429&partnerID=8YFLogxK

U2 - 10.1109/TMI.2016.2621185

DO - 10.1109/TMI.2016.2621185

M3 - Article

AN - SCOPUS:85012110429

SN - 0278-0062

VL - 36

SP - 674

EP - 683

JO - IEEE Transactions on Medical Imaging

JF - IEEE Transactions on Medical Imaging

IS - 2

M1 - 7739993

ER -

DeepCut: Object Segmentation from Bounding Box Annotations Using Convolutional Neural Networks

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this