Check Dose-Response Patterns

The idea behind the dose-response option is that causal inference is strengthened if the outcome condition improves as an increasing function of the amount of program participation. (Reynolds, 1998)

Example

The Oregon Healthy Start (OHS) was a voluntary primary prevention program designed to prevent child maltreatment. Families identified as being at risk for child maltreatment were offered home visits, parenting education, and extensive family support services by trained paraprofessionals. Home visits were offered weekly or biweekly, depending on the families’ needs and the counties’ caseload limitations.

The evaluator of the program examined a number of causal criteria including:

• Dose/ response
• Strength of associations
• Specificity of effects
• Consistency of association of program exposure and outcomes
• Coherence of outcomes

Mothers who had more exposure to the program were more aware of their prior lack of parenting knowledge and skills. These results supported the claim that the program produced the response shift. (Pratt, McGuigan & Katzev, 2000)

Advice

Advice for CHOOSING this option (tips and traps)

• Include this as a package of causal criteria to test the program theory. Remember that the absence of a dose/ response relationship does not rule out a causal relationship.

Advice for USING this option (tips and traps)

• Control for differences that could lead to different levels of program exposure.
• Consider nonlinear relationships e.g. more fertiliser does not always improve productivity - at some point it becomes excessive and toxic) and threshold effects (e.g. there's a theory in autism support that you need a certain number of hours of therapy a week to make a difference) that may alter patterns.
• Provide program theory documentation. Assist the evaluator develop a retrospective program theory if there is not one already developed.

Resources

Examples

• Methodologies to Evaluate the Impact of Large Scale Nutrition Programs: Page 30 - 31 of this report focuses on using dose-response analysis to strengthen causal inference in impact evaluation
• The South African Child Support Grant Impact Assessment: Appendix 5.2 of this report describes the methodology used to estimate dose response functions.

Source

Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341–349. Retrieved from http://aje.sagepub.com/content/21/3/341.full.pdf html

Reynolds, A. J. (1998). Confirmatory program evaluation: A option for strengthening causal inference. American Journal of Evaluation, 19(2), 203-221. Retrieved from http://aje.sagepub.com/content/19/2/203.full.pdf