King's College London

Research portal

Understanding the Replication Crisis as a Base Rate Fallacy

Research output: Contribution to journalArticle

Original languageEnglish
Article numberaxy051
JournalBRITISH JOURNAL FOR THE PHILOSOPHY OF SCIENCE
Early online date13 Aug 2018
DOIs
Publication statusE-pub ahead of print - 13 Aug 2018

King's Authors

Abstract

The replication (replicability, reproducibility) crisis in social psychology and clinical medicine arises from the fact that many apparently well-confirmed experimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this paper I argue that the high rate of failed replications is consistent with high quality science. We would expect this outcome if the field of science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null-hypothesis significance tests with a type-I error rate (α) of 5%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of science.

View graph of relations

© 2018 King's College London | Strand | London WC2R 2LS | England | United Kingdom | Tel +44 (0)20 7836 5454