Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data

Mark Eastwood; Silviu Tudor Marc; Xiaohong Gao; Heba Sailem; Judith Offman; Emmanouil Karteris; Angeles Montero Fernandez; Danny Jonigk; William Cookson; Miriam Moffatt; Sanjay Popat; Fayyaz Minhas; Jan Lukas Robertus

doi:10.1016/j.artmed.2023.102628

Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data

Mark Eastwood^*, Silviu Tudor Marc, Xiaohong Gao, Heba Sailem, Judith Offman, Emmanouil Karteris, Angeles Montero Fernandez, Danny Jonigk, William Cookson, Miriam Moffatt, Sanjay Popat, Fayyaz Minhas, Jan Lukas Robertus

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Malignant Mesothelioma is a difficult to diagnose and highly lethal cancer usually associated with asbestos exposure. It can be broadly classified into three subtypes: Epithelioid, Sarcomatoid, and a hybrid Biphasic subtype in which significant components of both of the previous subtypes are present. Early diagnosis and identification of the subtype informs treatment and can help improve patient outcome. However, the subtyping of malignant mesothelioma, and specifically the recognition of transitional features from routine histology slides has a high level of inter-observer variability. In this work, we propose an end-to-end multiple instance learning (MIL) approach for malignant mesothelioma subtyping. This uses an adaptive instance-based sampling scheme for training deep convolutional neural networks on bags of image patches that allows learning on a wider range of relevant instances compared to max or top-N based MIL approaches. We also investigate augmenting the instance representation to include aggregate cellular morphology features from cell segmentation. The proposed MIL approach enables identification of malignant mesothelial subtypes of specific tissue regions. From this a continuous characterisation of a sample according to predominance of sarcomatoid vs epithelioid regions is possible, thus avoiding the arbitrary and highly subjective categorisation by currently used subtypes. Instance scoring also enables studying tumor heterogeneity and identifying patterns associated with different subtypes. We have evaluated the proposed method on a dataset of 234 tissue micro-array cores with an AUROC of 0.89±0.05 for this task. The dataset and developed methodology is available for the community at: https://github.com/measty/PINS.

Original language	English
Article number	102628
Journal	Artificial Intelligence in Medicine
Volume	143
DOIs	https://doi.org/10.1016/j.artmed.2023.102628
Publication status	Published - Sept 2023

Keywords

Cancer subtyping
Computational pathology
Deep learning
Malignant Mesothelioma
Multiple instance learning

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.artmed.2023.102628

Cite this

Eastwood, M., Marc, S. T., Gao, X., Sailem, H., Offman, J., Karteris, E., Fernandez, A. M., Jonigk, D., Cookson, W., Moffatt, M., Popat, S., Minhas, F., & Robertus, J. L. (2023). Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data. Artificial Intelligence in Medicine, 143, Article 102628. https://doi.org/10.1016/j.artmed.2023.102628

@article{759b9c160ee346ddb79f32b91c78f70e,

title = "Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data",

abstract = "Malignant Mesothelioma is a difficult to diagnose and highly lethal cancer usually associated with asbestos exposure. It can be broadly classified into three subtypes: Epithelioid, Sarcomatoid, and a hybrid Biphasic subtype in which significant components of both of the previous subtypes are present. Early diagnosis and identification of the subtype informs treatment and can help improve patient outcome. However, the subtyping of malignant mesothelioma, and specifically the recognition of transitional features from routine histology slides has a high level of inter-observer variability. In this work, we propose an end-to-end multiple instance learning (MIL) approach for malignant mesothelioma subtyping. This uses an adaptive instance-based sampling scheme for training deep convolutional neural networks on bags of image patches that allows learning on a wider range of relevant instances compared to max or top-N based MIL approaches. We also investigate augmenting the instance representation to include aggregate cellular morphology features from cell segmentation. The proposed MIL approach enables identification of malignant mesothelial subtypes of specific tissue regions. From this a continuous characterisation of a sample according to predominance of sarcomatoid vs epithelioid regions is possible, thus avoiding the arbitrary and highly subjective categorisation by currently used subtypes. Instance scoring also enables studying tumor heterogeneity and identifying patterns associated with different subtypes. We have evaluated the proposed method on a dataset of 234 tissue micro-array cores with an AUROC of 0.89±0.05 for this task. The dataset and developed methodology is available for the community at: https://github.com/measty/PINS.",

keywords = "Cancer subtyping, Computational pathology, Deep learning, Malignant Mesothelioma, Multiple instance learning",

author = "Mark Eastwood and Marc, {Silviu Tudor} and Xiaohong Gao and Heba Sailem and Judith Offman and Emmanouil Karteris and Fernandez, {Angeles Montero} and Danny Jonigk and William Cookson and Miriam Moffatt and Sanjay Popat and Fayyaz Minhas and Robertus, {Jan Lukas}",

note = "Publisher Copyright: {\textcopyright} 2023 The Authors",

year = "2023",

month = sep,

doi = "10.1016/j.artmed.2023.102628",

language = "English",

volume = "143",

journal = "Artificial Intelligence in Medicine",

issn = "0933-3657",

publisher = "Elsevier",

}

Eastwood, M, Marc, ST, Gao, X, Sailem, H , Offman, J, Karteris, E, Fernandez, AM, Jonigk, D, Cookson, W, Moffatt, M, Popat, S, Minhas, F & Robertus, JL 2023, 'Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data', Artificial Intelligence in Medicine, vol. 143, 102628. https://doi.org/10.1016/j.artmed.2023.102628

TY - JOUR

T1 - Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data

AU - Eastwood, Mark

AU - Marc, Silviu Tudor

AU - Gao, Xiaohong

AU - Sailem, Heba

AU - Offman, Judith

AU - Karteris, Emmanouil

AU - Fernandez, Angeles Montero

AU - Jonigk, Danny

AU - Cookson, William

AU - Moffatt, Miriam

AU - Popat, Sanjay

AU - Minhas, Fayyaz

AU - Robertus, Jan Lukas

PY - 2023/9

Y1 - 2023/9

N2 - Malignant Mesothelioma is a difficult to diagnose and highly lethal cancer usually associated with asbestos exposure. It can be broadly classified into three subtypes: Epithelioid, Sarcomatoid, and a hybrid Biphasic subtype in which significant components of both of the previous subtypes are present. Early diagnosis and identification of the subtype informs treatment and can help improve patient outcome. However, the subtyping of malignant mesothelioma, and specifically the recognition of transitional features from routine histology slides has a high level of inter-observer variability. In this work, we propose an end-to-end multiple instance learning (MIL) approach for malignant mesothelioma subtyping. This uses an adaptive instance-based sampling scheme for training deep convolutional neural networks on bags of image patches that allows learning on a wider range of relevant instances compared to max or top-N based MIL approaches. We also investigate augmenting the instance representation to include aggregate cellular morphology features from cell segmentation. The proposed MIL approach enables identification of malignant mesothelial subtypes of specific tissue regions. From this a continuous characterisation of a sample according to predominance of sarcomatoid vs epithelioid regions is possible, thus avoiding the arbitrary and highly subjective categorisation by currently used subtypes. Instance scoring also enables studying tumor heterogeneity and identifying patterns associated with different subtypes. We have evaluated the proposed method on a dataset of 234 tissue micro-array cores with an AUROC of 0.89±0.05 for this task. The dataset and developed methodology is available for the community at: https://github.com/measty/PINS.

AB - Malignant Mesothelioma is a difficult to diagnose and highly lethal cancer usually associated with asbestos exposure. It can be broadly classified into three subtypes: Epithelioid, Sarcomatoid, and a hybrid Biphasic subtype in which significant components of both of the previous subtypes are present. Early diagnosis and identification of the subtype informs treatment and can help improve patient outcome. However, the subtyping of malignant mesothelioma, and specifically the recognition of transitional features from routine histology slides has a high level of inter-observer variability. In this work, we propose an end-to-end multiple instance learning (MIL) approach for malignant mesothelioma subtyping. This uses an adaptive instance-based sampling scheme for training deep convolutional neural networks on bags of image patches that allows learning on a wider range of relevant instances compared to max or top-N based MIL approaches. We also investigate augmenting the instance representation to include aggregate cellular morphology features from cell segmentation. The proposed MIL approach enables identification of malignant mesothelial subtypes of specific tissue regions. From this a continuous characterisation of a sample according to predominance of sarcomatoid vs epithelioid regions is possible, thus avoiding the arbitrary and highly subjective categorisation by currently used subtypes. Instance scoring also enables studying tumor heterogeneity and identifying patterns associated with different subtypes. We have evaluated the proposed method on a dataset of 234 tissue micro-array cores with an AUROC of 0.89±0.05 for this task. The dataset and developed methodology is available for the community at: https://github.com/measty/PINS.

KW - Cancer subtyping

KW - Computational pathology

KW - Deep learning

KW - Malignant Mesothelioma

KW - Multiple instance learning

UR - http://www.scopus.com/inward/record.url?scp=85166017188&partnerID=8YFLogxK

U2 - 10.1016/j.artmed.2023.102628

DO - 10.1016/j.artmed.2023.102628

M3 - Article

C2 - 37673586

AN - SCOPUS:85166017188

SN - 0933-3657

VL - 143

JO - Artificial Intelligence in Medicine

JF - Artificial Intelligence in Medicine

M1 - 102628

ER -

Malignant Mesothelioma subtyping via sampling driven multiple instance prediction on tissue image and cell morphology data

Abstract

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this