Using experiments in comparative policy analysis: from policy evaluation to the policy process

Peter John*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

1 Citation (Scopus)


This chapter sets out the advantages (and some disadvantages) of random allocation or the randomized controlled trial (RCT) as a method to test interventions and evaluate casual processes in the social sciences. Randomization allows the evaluation of the counterfactual as both observed and unobserved factors are equally present in both treatment and control groups except for the variable of interest or intervention. While RCTs were present at the founding of the social sciences in the early twentieth century, the method languished until its enthusiastic rediscovery by development economists and students of political behaviour in the late 1990s and early 2000s. Now both economics and political science have a profusion of experiments and many scholars adopt the framework of causal inference and use identified designs in the form of regression discontinuity designs, difference-in-differences, comparative interrupted time series, and instrumental variable models, which may be called natural experiments. Trials have been used increasingly in public policy evaluations since the 1960s. But studies of the policy process and implementation remain largely untouched. This contrasts with their rapid adoption in the sister discipline of public administration, which shares an interest in decision making and the role of institutions and organizations, as shown by a recent wave of experiments with bureaucrats and studies under the banner of behavioural public administration. The chapter seeks to explain this lack of interest as a result of social networks and traditions of study in public policy, which have led the predominance of case study and interview-based methods (which can of course be integrated into randomized designs). The conclusion to draw is that there is a massive gap to be filled in the study of public policy. Experimental studies should be carried out so public policy studies can keep pace with advances in other disciplines.

Original languageEnglish
Title of host publicationHandbook of Research Methods and Applications in Comparative Policy Analysis
PublisherEdward Elgar Publishing Ltd
Number of pages14
ISBN (Electronic)9781788111195
ISBN (Print)9781788111188
Publication statusPublished - 1 Jan 2020


Dive into the research topics of 'Using experiments in comparative policy analysis: from policy evaluation to the policy process'. Together they form a unique fingerprint.

Cite this