Are RCTs (Randomised Controlled Trials) a new approach in evaluation?

There is a long history of using RCTs to inform public policy decisions, especially in health,  but it is only more recently that they have been used widely in development evaluation.

James Lind is credited with being the father of clinical trials when, in 1747, he used a scientific experiment to demonstrate the effect of citrus fruits on scurvy prevention. [1].

In the 1920s, Neyman and Fisher conducted randomized trials on agricultural experiments, while, in the 1940s, with the publication of Sir Austin Bradford Hill’s paper, "Streptomycin treatment of pulmonary tuberculosis” in the British Medical Journal [2], RCTs soon became the standard in clinical research throughout the world.

Randomized impact evaluations branched out into social science research. For example, in 1971, RAND used an RCT approach in its well-known Health Insurance Experiment (HIE)[3] to investigate healthcare utilization and health outcomes in the United States. Social scientists used RCTs to evaluate programs completely outside of the health sector: for example, the New Jersey Maintenance Experiment, which measured the effect of welfare programs on poor households’ annual earnings [4].

An early use of RCTs in evaluating social programs was the PROGRESA program (now known as Oportunidades). PROGRESA was a conditional cash transfer (CCT) program implemented by the Mexican government in 1997 and evaluated by IFPRI researchers from both a process evaluation and impact evaluation stand-point. Due to budget constraints, PROGRESA was expanded sequentially to rural localities throughout the country. IFPRI researchers could thus compare localities which had been randomly assigned to receive the program in 1997, to those who would receive it in later years [5].

Contemporary efforts were being made by researchers at the Harvard Kennedy School (specifically, Michael Kremer) in using the random distribution of, for example, textbooks to determine their impact on children’s test scores in Kenya [6]. In 2003, the Abdul Latif Jameel Poverty Action Lab was founded – a research institute dedicated to using the RCT methodology in the evaluation of social programs.

Approach

This FAQ relates to the approach Randomized Controlled Trial (RCT).

Sources

[1] Thomas, D. P. (1997). Sailors, scurvy and science. Journal of the Royal Society of Medicine, 90(1), 50. http://europepmc.org/articles/PMC1296121/pdf/jrsocmed00044-0056.pdf

[2] Brown, D. (1998) "Landmark Study Made Research Resistant to Bias". Washington Post, 2 November 1998.  http://www.bsos.umd.edu/socy/alan/stats/Bias_Study.html

[3] RAND Archive. RAND's Health Insurance Experiment (HIE) http://www.rand.org/health/projects/hie.html

[4] Hausman, J. A., & Wise, D. A. (1976). The evaluation of results from truncated samples: The New Jersey income maintenance experiment. In Annals of Economic and Social Measurement, Volume 5, number 4 (pp. 421-445). NBER http://www.nber.org/chapters/c10489.pdf

[5] Skoufias, E. (2005). PROGRESA and its impacts on the welfare of rural households in Mexico (Vol. 139). Intl Food Policy Res Inst http://www.ifpri.org/sites/default/files/publications/rr139.pdf

[6] Glewwe, P., Kremer, M., & Moulin, S. (2007). Many children left behind? Textbooks and test scores in Kenya (No. w13300). National Bureau of Economic Research. http://www.nber.org/papers/w13300

Page Credits

Author: Reviewer: Steward: Other Contributors:
Angela Ambroz, Marc Shotland - - -