Are treatment effect assumptions in orthodontic studies overoptimistic?

Jadbinder Seehra, Daniel Stonehouse-Smith, Martyn T. Cobourne, Michail Tsagris, Nikolaos Pandis

Research output: Contribution to journalArticlepeer-review


Background: At the clinical trial design stage, assumptions regarding the treatment effects to be detected should be appropriate so that the required sample size can be calculated. There is evidence in the medical literature that sample size assumption can be overoptimistic. The aim of this study was to compare the distribution of the assumed effects versus that of the observed effects as a proxy for overoptimistic treatment effect assumptions at the study design stage. Materials and method: Systematic reviews (SRs) published between 1 January 2010 and 31 December 2019 containing at least one meta-analysis on continuous outcomes were identified electronically. SR and primary study level characteristics were extracted from the SRs and the individual trials. Details on the sample size calculation process and assumptions and the observed treatment effects were extracted. Results: Eighty-five SRs with meta-analysis containing 347 primary trials were included. The median number of SR authors was 5 (interquartile range: 4-7). At the primary study level, the majority were single centre (78.1%), utilized a parallel design (52%), and rated as an unclear/moderate level of risk of bias (34.3%). A sample size was described in only 31.7% (110/347) of studies. From this cohort of 110 studies, in only 37 studies was the assumed clinical difference that the study was designed to detect reported (37/110). The assumed treatment effect was recalculated for the remaining 73 studies (73/110). The one-sided exact signed rank test showed a significant difference between the assumed and observed treatment effects (P < 0.001) suggesting greater values for the assumed effect sizes. Conclusions: Careful consideration of the assumptions at the design stage of orthodontic studies are necessary in order to reduce the unreliability of clinical study results and research waste.

Original languageEnglish
Pages (from-to)583-587
Number of pages5
JournalEuropean Journal of Orthodontics
Issue number5
Early online date15 May 2021
Publication statusPublished - 1 Oct 2021


Dive into the research topics of 'Are treatment effect assumptions in orthodontic studies overoptimistic?'. Together they form a unique fingerprint.

Cite this