Regression discontinuity

Synonyms:
RD design, Regression discontinuity design

Regression Discontinuity Design (RDD) is a quasi-experimental evaluation option that measures the impact of an intervention, or treatment, by applying a treatment assignment mechanism based on a continuous eligibility index which is a variable with a continuous distribution.

For example, test scores of students can take any value between 0-100 and area of land owned can have infinite values. If treatment is assigned to those either above or below a certain “cut-off” point, RDD can be used to measure the difference in outcomes of individuals, households, or communities clustered around the defined cut-off point. This necessitates the determination of a ‘bandwidth’ around the cut-off point within which individual units are shown to be statistically comparable. If the assumptions of the option hold, the difference in outcomes between those above and below the cut-off can be attributed to the program.

Examples

School Enrollment, Selection and Test Scores

The study (see table below) analyzed the impact of a scholarship program, the Cambodia Education Sector Support Program (CESSP, archived link), on school enrollment, selection, and test scores using a sharp regression discontinuity design (RDD). Eligibility for the $45 scholarship depended on an independently calculated dropout risk score created using household characteristics. The data used included the composite dropout risk scores, mathematics and vocabulary test scores, and a household survey. The cut-off score varied by school size, and the estimates were weighted averages. The sample used in the analysis consisted of children within a ten-point bandwidth range around the cut-off score. The evaluation found that school enrollment and attendance increased 20-25% due to the scholarships. Years of schooling increased by 0.21 years, while annual educational expenses paid by the family rose by $9. The actual program impact was likely to be higher since recipients were, on average, poorer than non-recipients. No significant impacts on learning outcomes were found.  However, recipients had better knowledge of HIV/AIDS, and the program had a positive effect on the recipients’ mental health.  (Filmer & Schady, 2009)

Assumptions:

  • Treatment is assigned on the basis of an observable variable or index
  • There is a discontinuity in the probability of being treated at some cut-off value of the assignment index or variable
  • The cut-off value of the index for treatment is arbitrary, and thus units on either side of the cut-off point are identical, on average, except for the presence of the treatment
  • The same cutoff is not used for determining eligibility for multiple treatments (e.g. households below a certain poverty line have access to multiple government interventions)

Program effects on school attendance

Example scatter plot with trend lines

Two Versions:

1. Sharp
  • Used in instances where treatment is assigned according to a discrete cut-off point with all eligible receiving the treatment and all ineligible not receiving it. A simple comparison of means is needed to calculate the effect of the treatment.   
2. Fuzzy
  • Used in instances where some eligible fail to receive treatment and/or ineligible receive it (e.g. due to self-selection or administrative overrides). The fuzzy version is more often utilized in practice. 
  • To calculate the effect of the treatment: a) Regress observable characteristics on outcomes to obtain the ‘outcome discontinuity', and b) Regress observable characteristics on the treatment indicator to obtain ‘treatment discontinuity.’ Then, Treatment effect = outcome discontinuity / treatment discontinuity

Advice for choosing this method

RDD can be used in situations where:

  • A continuous eligibility index (e.g. test scores, age etc.) is used to rank the population and
  • A clearly defined cut-off point that determines eligibility for treatment is part of the program design and
  • The same cut-off score has not been used in determining eligibility for other treatments.
  • Be aware that RD provides limited external validity as results are only generalizable around the cut-off - provision of a service might make more or less difference to people who are further away from the cut-off point.

Advice for using this method

  • Test for the existence of a discontinuity in the probability of receiving treatment (Nichols, 2009)
  • Test for the comparability of units within the bandwidth using ‘leave-one-out’ cross validation test (Ludwig and Miller 2007, Imbens and Lemieux 2008) or asymptotic theory (Imbens and Kalyanaraman 2009).
  • Depending on the type of treatment there may be ‘cross-contamination’ between those eligible for the intervention and those who are not.

Resource

Buddelmeyer, H., & Skoufias, E. World Bank, (2004). An evaluation of the performance of regression discontinuity design on PROGRESA& (World Bank Policy Research Working Paper 3386). Retrieved from website: https://openknowledge.worldbank.org/handle/10986/14153

Campbell, D. T. (1971). Reforms as experiments. Urban Affairs Review, 7, 133-171. Retrieved from http://uar.sagepub.com/content/7/2/133.full.pdf html

Filmer, D., & Schady, N. The World Bank, Human Development and Public Services Team. (2009). School enrollment, selection and test scores (Policy Research Working Paper 4998). Retrieved from website: https://openknowledge.worldbank.org/bitstream/handle/10986/4190/WPS4998.pdf

Hahn, J., Todd, P., & Van der Klaauw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica, 69(1), 201-209. 

Imbens, G., & Lemieux, T. (2007). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615-635. Retrieved from https://scholar.harvard.edu/files/imbens/files/regression_discontinuity_designs_a_guide_to_practice.pdf.

Khandker, S. R., Koolwal, G. B., & Samad, H. A. (2010). Handbook on impact evaluation: Quantitative methods and practices (52099). World Bank. Retrieved from website: https://openknowledge.worldbank.org/handle/10986/2693

Nichols, A. (2009). Causal inference with observational data regression discontinuity and related options in STATA. STATA. Retrieved from website: http://www.stata.com/meeting/germany09/nichols.pdf

Expand to view all resources related to 'Regression discontinuity'

'Regression discontinuity' is referenced in: