Conditions to consider in the use of randomized experimental designs in evaluation

This paper, written by George Julnes, University of New Mexico, Melvin M. Mark, Penn State University, and Stephanie Shipman, U.S. Government Accountability Office (retired), discusses the role of randomised control trials (RCT) in evaluation.

The paper intends to provide a non-technical summary of general factors to consider when determining the appropriateness of a randomised design in a forthcoming evaluation or set of evaluations. The paper proposes four general conditions that should be taken into account, concerning the specific context for the upcoming evaluation(s), when deciding whether or not an RCT is appropriate:

• The expected value of the information that a well-implemented experiment can provide in the specific context;
• the legal and ethical issues that apply in the circumstances at hand;
• the practical constraints (or facilitating factors) that would apply to a randomized experiment in that context, and
• the likely value of the experimental findings in relation to and as part of a portfolio of evaluative studies in the specific context.

The paper is intended to support thoughtful judgments about whether and when to use a randomized experimental design, especially by funders and others who influence the choice of an evaluation design but are not immersed in the methodological literature.