King's College London

Background: A prediction interval represents a clinical interpretation of heterogeneity. The aim of this study was to determine the prevalence of prediction interval reporting in orthodontic random effect meta-analyses. The corroboration between effect size estimates with 95% confidence intervals (CIs) and prediction intervals were also explored.

Materials and methods: Systematic reviews (SRs) published between 1 January 2010 and 31 January 2021 containing at least one random effects meta-analysis (minimum of three trials) were identified electronically. SR and meta-analyses characteristics were extracted and prediction intervals, where possible, were calculated. Descriptive statistics and the percentage of meta-analyses where the prediction interval changed the interpretation based on the 95% CI were calculated. Fisher's exact test was used to examine associations between the study variables and reporting of prediction intervals.

Results: One hundred and twenty-one SRs were included. The median number of SR authors was 5 (interquartile range: 4-6). The reporting of prediction intervals was undertaken in only 19.0% (N = 23/121) of meta-analyses. Out of 95 meta-analyses, only in 6 (6.3%, N = 6/95) were the 95% CI corroborated by the prediction interval. In 60 meta-analyses (63.3%, N = 60/95) despite a 95% CI indicating a statistically significant result, this was not corroborated by the corresponding prediction interval.

Conclusions: Within the study timeframe, reporting of prediction intervals is not routinely undertaken in orthodontic meta-analyses possibly due to a lack of awareness. In future orthodontic random effects models containing a minimum of three trials, reporting of prediction intervals is advocated as this gives an indication of the range of the expected effect of treatment interventions.