TY - JOUR
T1 - Benchmarking framework for myocardial tracking and deformation algorithms
T2 - An open access database
AU - Tobon-Gomez, C.
AU - De Craene, M.
AU - McLeod, K.
AU - Tautz, L.
AU - Shi, W.
AU - Hennemuth, A.
AU - Prakosa, A.
AU - Wang, H.
AU - Carr-White, Gerald
AU - Kapetanakis, Stamatis
AU - Lutz, A.
AU - Rasche, V.
AU - Schaeffter, T.
AU - Butakoff, C.
AU - Friman, O.
AU - Mansi, T.
AU - Sermesant, M.
AU - Zhuang, X.
AU - Ourselin, S.
AU - Peitgen, H-O.
AU - Pennec, X.
AU - Razavi, R.
AU - Rueckert, D.
AU - Frangi, A. F.
AU - Rhode, K. S.
PY - 2013/8
Y1 - 2013/8
N2 - In this paper we present a benchmarking framework for the validation of cardiac motion analysis algorithms. The reported methods are the response to an open challenge that was issued to the medical imaging community through a MICCAI workshop. The database included magnetic resonance (MR) and 3D ultrasound (3DUS) datasets from a dynamic phantom and 15 healthy volunteers. Participants processed 3D tagged MR datasets (3DTAG), cine steady state free precession MR datasets (SSFP) and 3DUS datasets, amounting to 1158 image volumes. Ground-truth for motion tracking was based on 12 landmarks (4 walls at 3 ventricular levels). They were manually tracked by two observers in the 3DTAG data over the whole cardiac cycle, using an in-house application with 4D visualization capabilities. The median of the inter-observer variability was computed for the phantom dataset (0.77 mm) and for the volunteer datasets (0.84 mm). The ground-truth was registered to 3DUS coordinates using a point based similarity transform. Four institutions responded to the challenge by providing motion estimates for the data: Fraunhofer MEVIS (MEVIS), Bremen, Germany; Imperial College London - University College London (IUCL), UK; Universitat Pompeu Fabra (UPF), Barcelona, Spain; Inria-Asclepios project (INRIA), France. Details on the implementation and evaluation of the four methodologies are presented in this manuscript. The manually tracked landmarks were used to evaluate tracking accuracy of all methodologies. For 3DTAG, median values were computed over all time frames for the phantom dataset (MEVIS = 1.20 mm, IUCL = 0.73 mm, UPF = 1.10 mm, INRIA = 1.09 mm) and for the volunteer datasets (MEVIS = 1.33 mm, IUCL = 1.52 mm, UPF = 1.09 mm, INRIA = 1.32 mm). For 3DUS, median values were computed at end diastole and end systole for the phantom dataset (MEVIS = 4.40 mm, UPF = 3.48 mm, INRIA = 4.78 mm) and for the volunteer datasets (MEVIS = 3.51 mm, UPF = 3.71 mm, INRIA = 4.07 mm). For SSFP, median values were computed at end diastole and end systole for the phantom dataset (UPF = 6.18 mm, INRIA = 3.93 mm) and for the volunteer datasets (UPF = 3.09 mm, INRIA = 4.78 mm). Finally, strain curves were generated and qualitatively compared. Good agreement was found between the different modalities and methodologies, except for radial strain that showed a high variability in cases of lower image quality.
AB - In this paper we present a benchmarking framework for the validation of cardiac motion analysis algorithms. The reported methods are the response to an open challenge that was issued to the medical imaging community through a MICCAI workshop. The database included magnetic resonance (MR) and 3D ultrasound (3DUS) datasets from a dynamic phantom and 15 healthy volunteers. Participants processed 3D tagged MR datasets (3DTAG), cine steady state free precession MR datasets (SSFP) and 3DUS datasets, amounting to 1158 image volumes. Ground-truth for motion tracking was based on 12 landmarks (4 walls at 3 ventricular levels). They were manually tracked by two observers in the 3DTAG data over the whole cardiac cycle, using an in-house application with 4D visualization capabilities. The median of the inter-observer variability was computed for the phantom dataset (0.77 mm) and for the volunteer datasets (0.84 mm). The ground-truth was registered to 3DUS coordinates using a point based similarity transform. Four institutions responded to the challenge by providing motion estimates for the data: Fraunhofer MEVIS (MEVIS), Bremen, Germany; Imperial College London - University College London (IUCL), UK; Universitat Pompeu Fabra (UPF), Barcelona, Spain; Inria-Asclepios project (INRIA), France. Details on the implementation and evaluation of the four methodologies are presented in this manuscript. The manually tracked landmarks were used to evaluate tracking accuracy of all methodologies. For 3DTAG, median values were computed over all time frames for the phantom dataset (MEVIS = 1.20 mm, IUCL = 0.73 mm, UPF = 1.10 mm, INRIA = 1.09 mm) and for the volunteer datasets (MEVIS = 1.33 mm, IUCL = 1.52 mm, UPF = 1.09 mm, INRIA = 1.32 mm). For 3DUS, median values were computed at end diastole and end systole for the phantom dataset (MEVIS = 4.40 mm, UPF = 3.48 mm, INRIA = 4.78 mm) and for the volunteer datasets (MEVIS = 3.51 mm, UPF = 3.71 mm, INRIA = 4.07 mm). For SSFP, median values were computed at end diastole and end systole for the phantom dataset (UPF = 6.18 mm, INRIA = 3.93 mm) and for the volunteer datasets (UPF = 3.09 mm, INRIA = 4.78 mm). Finally, strain curves were generated and qualitatively compared. Good agreement was found between the different modalities and methodologies, except for radial strain that showed a high variability in cases of lower image quality.
KW - Cardiac motion tracking
KW - 3D tagged MR
KW - 3D ultrasound
KW - Multimodal
KW - Spatiotemporal registration
KW - LEFT-VENTRICULAR DEFORMATION
KW - B-SPLINE MODEL
KW - NONRIGID REGISTRATION
KW - TAGGED MRI
KW - AUTOMATIC SEGMENTATION
KW - IMAGE REGISTRATION
KW - STRAIN ESTIMATION
KW - ULTRASOUND
KW - MOTION
KW - QUANTIFICATION
U2 - 10.1016/j.media.2013.03.008
DO - 10.1016/j.media.2013.03.008
M3 - Article
SN - 1361-8415
VL - 17
SP - 632
EP - 648
JO - Medical Image Analysis
JF - Medical Image Analysis
IS - 6
ER -