Evaluators can examine the link between dose and response as part of determining whether the program caused the outcome.
The program theory will identify the key expected responses and outcomes of the program by domain. Develop a retrospective program theory with expected responses if this has not been done already.
The idea behind the dose-response method is that causal inference is strengthened if the outcome condition improves as an increasing function of the amount of program participation. (Reynolds, 1998)
The Oregon Healthy Start (OHS) was a voluntary primary prevention program designed to prevent child maltreatment. Families identified as being at risk for child maltreatment were offered home visits, parenting education, and extensive family support services by trained paraprofessionals. Home visits were offered weekly or biweekly, depending on the families’ needs and the counties’ caseload limitations.
The evaluator of the program examined a number of causal criteria including:
- Dose/ response
- Strength of associations
- Specificity of effects
- Consistency of association of program exposure and outcomes
- Coherence of outcomes
Mothers who had more exposure to the program were more aware of their prior lack of parenting knowledge and skills. These results supported the claim that the program produced the response shift. (Pratt, McGuigan & Katzev, 2000)
Advice for choosing this method
Include this as a package of causal criteria to test the program theory. Remember that the absence of a dose/ response relationship does not rule out a causal relationship.
Advice for using this method
- Control for differences that could lead to different levels of program exposure.
- Consider nonlinear relationships e.g. more fertiliser does not always improve productivity - at some point it becomes excessive and toxic) and threshold effects (e.g. there's a theory in autism support that you need a certain number of hours of therapy a week to make a difference) that may alter patterns.
- Provide program theory documentation. Assist the evaluator to develop a retrospective program theory if there is not one already developed.
- Page 30 - 31 of this report focuses on using dose-response analysis to strengthen causal inference in impact evaluation
- Appendix 5.2 of this report describes the methodology used to estimate dose-response functions.
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341–349. Retrieved from https://journals.sagepub.com/doi/pdf/10.1177/109821400002100305
Reynolds, A. J. (1998). Confirmatory program evaluation: A option for strengthening causal inference. American Journal of Evaluation, 19(2), 203-221. Retrieved from https://journals.sagepub.com/doi/pdf/10.1177/109821409801900204
'Check dose-response patterns' is referenced in:
- Communication for Development (C4D) :
- Rainbow Framework :