The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

Alex  Zwanenburg; Martin Vallieres; Mahmoud Abdalah; Hugo Aerts; Vincent Andrearczyk; Aditya Apte; Saeed Ashrafinia; Spyridon Bakas; Roelof Beukinga; Ronald Boellaard; Marta Bogowicz; Luca Boldrini; Irene Buvat; Gary Cook; Christos Davatzikos; Adrien Depeursinge; Marie-Charlotte Desseroit; Nicola Dinapoli; Cuong Viet Dinh; Sebastien Echegaray; Issam El naqa; Andriy Fedorov; Roberto Gatta; Robert Gillies; Vicky Goh; Michael Gotz; Matthias Guckenberger; Sung Min Ha; Matthieu Hatt; Fabian Isensee; Phillipe Lambin; Stefan Leger; Ralph Leijenaar; Jacopo Lenkowitz; Fiona Lippert; Are Losnegard; Klaus Maier-Hein; Olivier Morin; Henning Muller; Sandy  Napel; Christoph Nioche; Fanny Orlhac; Sarthak Pati; Elisabeth Pfaehler; Arman Rahmin; Arvind Rao; Jonas Scherer; Musib Siddique; Nanna Sijtsema; Jairo Socarras Fernandez; Emiliano Spezi; Roel Steenbakkers; Stephanie Tanadini-Lang; Daniela Thorwarth; Esther Troost; Taman Upadhaya; Vincenzo Valentini; Lisanne Van Dijk; Joost Van Griethuysen; Floris Van Velden; Philip Whybra; Chritian  Richter; Steffan Lock

doi:10.1148/radiol.2020191145

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

Alex Zwanenburg, Martin Vallieres, Mahmoud Abdalah, Hugo Aerts, Vincent Andrearczyk, Aditya Apte, Saeed Ashrafinia, Spyridon Bakas, Roelof Beukinga, Ronald Boellaard, Marta Bogowicz, Luca Boldrini, Irene Buvat, Gary Cook, Christos Davatzikos, Adrien Depeursinge, Marie-Charlotte Desseroit, Nicola Dinapoli, Cuong Viet Dinh, Sebastien EchegarayIssam El naqa, Andriy Fedorov, Roberto Gatta, Robert Gillies, Vicky Goh, Michael Gotz, Matthias Guckenberger, Sung Min Ha, Matthieu Hatt , Fabian Isensee, Phillipe Lambin, Stefan Leger, Ralph Leijenaar, Jacopo Lenkowitz, Fiona Lippert, Are Losnegard, Klaus Maier-Hein, Olivier Morin, Henning Muller, Sandy Napel, Christoph Nioche, Fanny Orlhac, Sarthak Pati, Elisabeth Pfaehler, Arman Rahmin, Arvind Rao, Jonas Scherer, Musib Siddique, Nanna Sijtsema, Jairo Socarras Fernandez, Emiliano Spezi, Roel Steenbakkers, Stephanie Tanadini-Lang, Daniela Thorwarth, Esther Troost, Taman Upadhaya, Vincenzo Valentini, Lisanne Van Dijk, Joost Van Griethuysen, Floris Van Velden, Philip Whybra, Chritian Richter, Steffan Lock

Cancer Imaging

Research output: Contribution to journal › Article › peer-review

2010 Citations (Scopus)

97 Downloads (Pure)

Abstract

Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.

Original language	English
Pages (from-to)	328-338
Number of pages	11
Journal	Radiology
Volume	295
Issue number	2
DOIs	https://doi.org/10.1148/radiol.2020191145
Publication status	Published - 1 May 2020

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1148/radiol.2020191145

Radiology_IBSI_Accepted_VersionAccepted author manuscript, 2.12 MBLicence: Unspecified

https://pubs.rsna.org/doi/10.1148/radiol.2020191145

Cite this

Zwanenburg, A., Vallieres, M., Abdalah, M., Aerts, H., Andrearczyk, V., Apte, A., Ashrafinia, S., Bakas, S., Beukinga, R., Boellaard, R., Bogowicz, M., Boldrini, L., Buvat, I., Cook, G., Davatzikos, C., Depeursinge, A., Desseroit, M.-C., Dinapoli, N., Dinh, C. V., ... Lock, S. (2020). The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. Radiology, 295(2), 328-338. https://doi.org/10.1148/radiol.2020191145

@article{dae7996002824ea196529ae0d7fbc26b,

title = "The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping",

abstract = "Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.",

author = "Alex Zwanenburg and Martin Vallieres and Mahmoud Abdalah and Hugo Aerts and Vincent Andrearczyk and Aditya Apte and Saeed Ashrafinia and Spyridon Bakas and Roelof Beukinga and Ronald Boellaard and Marta Bogowicz and Luca Boldrini and Irene Buvat and Gary Cook and Christos Davatzikos and Adrien Depeursinge and Marie-Charlotte Desseroit and Nicola Dinapoli and Dinh, {Cuong Viet} and Sebastien Echegaray and {El naqa}, Issam and Andriy Fedorov and Roberto Gatta and Robert Gillies and Vicky Goh and Michael Gotz and Matthias Guckenberger and Ha, {Sung Min} and Matthieu Hatt and Fabian Isensee and Phillipe Lambin and Stefan Leger and Ralph Leijenaar and Jacopo Lenkowitz and Fiona Lippert and Are Losnegard and Klaus Maier-Hein and Olivier Morin and Henning Muller and Sandy Napel and Christoph Nioche and Fanny Orlhac and Sarthak Pati and Elisabeth Pfaehler and Arman Rahmin and Arvind Rao and Jonas Scherer and Musib Siddique and Nanna Sijtsema and {Socarras Fernandez}, Jairo and Emiliano Spezi and Roel Steenbakkers and Stephanie Tanadini-Lang and Daniela Thorwarth and Esther Troost and Taman Upadhaya and Vincenzo Valentini and {Van Dijk}, Lisanne and {Van Griethuysen}, Joost and {Van Velden}, Floris and Philip Whybra and Chritian Richter and Steffan Lock",

year = "2020",

month = may,

day = "1",

doi = "10.1148/radiol.2020191145",

language = "English",

volume = "295",

pages = "328--338",

journal = "Radiology",

issn = "0033-8419",

publisher = "Radiological Society of North America",

number = "2",

}

Zwanenburg, A, Vallieres, M, Abdalah, M, Aerts, H, Andrearczyk, V, Apte, A, Ashrafinia, S, Bakas, S, Beukinga, R, Boellaard, R, Bogowicz, M, Boldrini, L, Buvat, I, Cook, G, Davatzikos, C, Depeursinge, A, Desseroit, M-C, Dinapoli, N, Dinh, CV, Echegaray, S, El naqa, I, Fedorov, A, Gatta, R, Gillies, R, Goh, V, Gotz, M, Guckenberger, M, Ha, SM, Hatt , M, Isensee, F, Lambin, P, Leger, S, Leijenaar, R, Lenkowitz, J, Lippert, F, Losnegard, A, Maier-Hein, K, Morin, O, Muller, H, Napel, S, Nioche, C, Orlhac, F, Pati, S, Pfaehler, E, Rahmin, A, Rao, A, Scherer, J, Siddique, M, Sijtsema, N, Socarras Fernandez, J, Spezi, E, Steenbakkers, R, Tanadini-Lang, S, Thorwarth, D, Troost, E, Upadhaya, T, Valentini, V, Van Dijk, L, Van Griethuysen, J, Van Velden, F, Whybra, P, Richter, C & Lock, S 2020, 'The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping', Radiology, vol. 295, no. 2, pp. 328-338. https://doi.org/10.1148/radiol.2020191145

TY - JOUR

T1 - The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

AU - Zwanenburg, Alex

AU - Vallieres, Martin

AU - Abdalah, Mahmoud

AU - Aerts, Hugo

AU - Andrearczyk, Vincent

AU - Apte, Aditya

AU - Ashrafinia, Saeed

AU - Bakas, Spyridon

AU - Beukinga, Roelof

AU - Boellaard, Ronald

AU - Bogowicz, Marta

AU - Boldrini, Luca

AU - Buvat, Irene

AU - Cook, Gary

AU - Davatzikos, Christos

AU - Depeursinge, Adrien

AU - Desseroit, Marie-Charlotte

AU - Dinapoli, Nicola

AU - Dinh, Cuong Viet

AU - Echegaray, Sebastien

AU - El naqa, Issam

AU - Fedorov, Andriy

AU - Gatta, Roberto

AU - Gillies, Robert

AU - Goh, Vicky

AU - Gotz, Michael

AU - Guckenberger, Matthias

AU - Ha, Sung Min

AU - Hatt , Matthieu

AU - Isensee, Fabian

AU - Lambin, Phillipe

AU - Leger, Stefan

AU - Leijenaar, Ralph

AU - Lenkowitz, Jacopo

AU - Lippert, Fiona

AU - Losnegard, Are

AU - Maier-Hein, Klaus

AU - Morin, Olivier

AU - Muller, Henning

AU - Napel, Sandy

AU - Nioche, Christoph

AU - Orlhac, Fanny

AU - Pati, Sarthak

AU - Pfaehler, Elisabeth

AU - Rahmin, Arman

AU - Rao, Arvind

AU - Scherer, Jonas

AU - Siddique, Musib

AU - Sijtsema, Nanna

AU - Socarras Fernandez, Jairo

AU - Spezi, Emiliano

AU - Steenbakkers, Roel

AU - Tanadini-Lang, Stephanie

AU - Thorwarth, Daniela

AU - Troost, Esther

AU - Upadhaya, Taman

AU - Valentini, Vincenzo

AU - Van Dijk, Lisanne

AU - Van Griethuysen, Joost

AU - Van Velden, Floris

AU - Whybra, Philip

AU - Richter, Chritian

AU - Lock, Steffan

PY - 2020/5/1

Y1 - 2020/5/1

N2 - Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.

AB - Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.

UR - http://www.scopus.com/inward/record.url?scp=85081738849&partnerID=8YFLogxK

U2 - 10.1148/radiol.2020191145

DO - 10.1148/radiol.2020191145

M3 - Article

SN - 0033-8419

VL - 295

SP - 328

EP - 338

JO - Radiology

JF - Radiology

IS - 2

ER -

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

Abstract

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this