Check Dose-Response Patterns

Evaluators can examine the link between dose and response as part of determining whether the program caused the outcome. The program theory will identify the key expected responses and outcomes of the program by domain. Develop a retrospective program theory with expected responses if this has not been done already.

The idea behind the dose-response option is that causal inference is strengthened if the outcome condition improves as an increasing function of the amount of program participation. (Reynolds, 1998)

Example

The Oregon Healthy Start (OHS) was a voluntary primary prevention program designed to prevent child maltreatment. Families identified as being at risk for child maltreatment were offered home visits, parenting education, and extensive family support services by trained paraprofessionals. Home visits were offered weekly or biweekly, depending on the families’ needs and the counties’ caseload limitations.

The evaluator of the program examined a number of causal criteria including:

  • Dose/ response
  • Strength of associations
  • Specificity of effects
  • Consistency of association of program exposure and outcomes
  • Coherence of outcomes

Mothers who had more exposure to the program were more aware of their prior lack of parenting knowledge and skills. These results supported the claim that the program produced the response shift. (Pratt, McGuigan & Katzev, 2000)

Advice

Advice for CHOOSING this option (tips and traps)

  • Include this as a package of causal criteria to test the program theory. Remember that the absence of a dose/ response relationship does not rule out a causal relationship.

Advice for USING this option (tips and traps)

  • Control for differences that could lead to different levels of program exposure.
  • Consider nonlinear relationships e.g. more fertiliser does not always improve productivity - at some point it becomes excessive and toxic) and threshold effects (e.g. there's a theory in autism support that you need a certain number of hours of therapy a week to make a difference) that may alter patterns.
  • Provide program theory documentation. Assist the evaluator develop a retrospective program theory if there is not one already developed.

Resources

Examples

Source

Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341–349. Retrieved from http://aje.sagepub.com/content/21/3/341.full.pdf html

Reynolds, A. J. (1998). Confirmatory program evaluation: A option for strengthening causal inference. American Journal of Evaluation, 19(2), 203-221. Retrieved from http://aje.sagepub.com/content/19/2/203.full.pdf

Updated: 14th January 2014 - 1:51am

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.