King's College London

Research portal

Knowledge Distillation for Semi-supervised Domain Adaptation

Research output: Chapter in Book/Report/Conference proceedingConference paper

Mauricio Orbes-Arteainst, Jorge Cardoso, Lauge Sørensen, Christian Igel, Sebastien Ourselin, Marc Modat, Mads Nielsen, Akshay Pai

Original languageEnglish
Title of host publicationOR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging - 2nd International Workshop, OR 2.0 2019, and 2nd International Workshop, MLCN 2019, Held in Conjunction with MICCAI 2019, Proceedings
EditorsLuping Zhou, Duygu Sarikaya, Seyed Mostafa Kia, Stefanie Speidel, Anand Malpani, Daniel Hashimoto, Mohamad Habes, Tommy Löfstedt, Kerstin Ritter, Hongzhi Wang
PublisherSPRINGER
Pages68-76
Number of pages9
ISBN (Print)9783030326944
DOIs
Publication statusPublished - 1 Jan 2019
Event2nd International Workshop on Context-Aware Surgical Theaters, OR 2.0 2019, and the 2nd International Workshop on Machine Learning in Clinical Neuroimaging, MLCN 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019 - Shenzhen, China
Duration: 17 Oct 201917 Oct 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11796 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference2nd International Workshop on Context-Aware Surgical Theaters, OR 2.0 2019, and the 2nd International Workshop on Machine Learning in Clinical Neuroimaging, MLCN 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2019
CountryChina
CityShenzhen
Period17/10/201917/10/2019

King's Authors

Abstract

In the absence of sufficient data variation (e.g., scanner and protocol variability) in annotated data, deep neural networks (DNNs) tend to overfit during training. As a result, their performance is significantly lower on data from unseen sources compared to the performance on data from the same source as the training data. Semi-supervised domain adaptation methods can alleviate this problem by tuning networks to new target domains without the need for annotated data from these domains. Adversarial domain adaptation (ADA) methods are a popular choice that aim to train networks in such a way that the features generated are domain agnostic. However, these methods require careful dataset-specific selection of hyperparameters such as the complexity of the discriminator in order to achieve a reasonable performance. We propose to use knowledge distillation (KD) – an efficient way of transferring knowledge between different DNNs – for semi-supervised domain adaption of DNNs. It does not require dataset-specific hyperparameter tuning, making it generally applicable. The proposed method is compared to ADA for segmentation of white matter hyperintensities (WMH) in magnetic resonance imaging (MRI) scans generated by scanners that are not a part of the training set. Compared with both the baseline DNN (trained on source domain only and without any adaption to target domain) and with using ADA for semi-supervised domain adaptation, the proposed method achieves significantly higher WMH dice scores.

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454