King's College London

Research portal

Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Research output: Contribution to journalArticle

Huseyin Naci, Courtney Davis, Jelena Savović, Julian P T Higgins, Jonathan A C Sterne, Bishal Gyawali, Xochitl Romo-Sandoval, Nicola Handley, Christopher M Booth

Original languageEnglish
Article numberl5221
Pages (from-to)l5221
JournalBMJ (Clinical research ed.)
Publication statusPublished - 18 Sep 2019

Bibliographical note

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to


King's Authors


OBJECTIVE: To examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials of cancer drugs approved by the European Medicines Agency (EMA).

DESIGN: Cross sectional analysis.

SETTING: European regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices.

ELIGIBILITY CRITERIA: Pivotal randomised controlled trials of new cancer drugs approved by the EMA between 2014 and 2016.

MAIN OUTCOME MEASURES: Study design characteristics (randomisation, comparators, and endpoints); risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result); and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents).

RESULTS: Between 2014 and 2016, the EMA approved 32 new cancer drugs on the basis of 54 pivotal studies. Of these, 41 (76%) were randomised controlled trials and 13 (24%) were either non-randomised studies or single arm studies. 39/41 randomised controlled trials had available publications and were included in our study. Only 10 randomised controlled trials (26%) measured overall survival as either a primary or coprimary endpoint, with the remaining trials evaluating surrogate measures such as progression free survival and response rates. Overall, 19 randomised controlled trials (49%) were judged to be at high risk of bias for their primary outcome. Concerns about missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer randomised controlled trials that evaluated overall survival as the primary endpoint were at high risk of bias than those that evaluated surrogate efficacy endpoints (2/10 (20%) v 16/29 (55%), respectively). When information available in regulatory documents and the scientific literature was considered separately, overall risk of bias judgments differed for eight randomised controlled trials (21%), which reflects reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These deficits included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications.

CONCLUSIONS: Most pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were randomised controlled trials. However, almost half of these were judged to be at high risk of bias based on their design, conduct, or analysis, some of which might be unavoidable because of the complexity of cancer trials. Regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454