Confirmatory Program Evaluation: A Method for Strengthening Causal Inference

This paper focusses on the analysis of current issues in theory driven evlauation as discussed in evaluation practice literature.

"Applications of theory-driven approaches are not easily found, in part, because there are few procedures for conducting them. I offer confirmatory program evaluation as one way to use theory, in combination with quantitative analytical techniques, to assess the effects of social and educational programs. In contrast to many other approaches, theory driven evaluation generally emphasizes the explication and testing of a priori program theories in determining effectiveness. Confirmatory program evaluation is an impact assessment that examines the pattern of empirical findings against several causal criteria, including temporality, size, gradient (dosage/response), specificity, consistency, and coherence of the program-outcome relationship. A special emphasis is given to identifying causal mechanisms or active ingredients of programs that yield effects. An illustration of confirmatory program evaluation is provided for a child development intervention called the Child Parent Center Program. The limitations of this option are discussed, as well as the conditions under which it is most useful." (Reynolds, 1998)

Contents

  • Lack of Theory-Based Evaluation in Practice
  • Nature of Causal Inference in Social Programs
  • Confirmatory Program Evaluation
  • Six Criteria for Interpreting Findings in Confirmatory Program Evaluation
  • The Role of Process Variables and Mediating Effects in Assessing Coherence of the Program-Outcome Relationship
  • Implementing a Confirmatory Program Evaluation
  • An Example of Confirmatory Program Evaluation From Child Development
  • Limitations of Confirmatory Program Evaluation

Specific Options Mentioned

  • Temporality of program exposure
  • Strength of association
  • Gradient effect (dosage/response).
  • Specificity
  • Consistency
  • Coherence

Sources

Reynolds, A. J. (1998). Confirmatory program evaluation: A option for strengthening causal inference. American Journal of Evaluation, 19(2), 203-221. Retrieved from http://aje.sagepub.com/content/19/2/203.full.pdf