Understand Causes

Most evaluations need to investigate what is causing the outcomes and impacts of an intervention. (Some process evaluations assume that certain activities are contributing to intended outcomes without investigating these).

Sometimes it is useful to think about this in terms of ‘causal attribution’ – did the intervention cause the outcomes and impacts that have been observed?  In many cases, however, the outcomes and impacts have been caused by a combination of programs, or by a program in combination with other factors.  

In such cases it can be more useful to think about “causal contribution” – did the intervention contribute to the outcomes and impacts that have been observed?

Tasks

1. Check the results are consistent with causal contribution

Check that the data are consistent with what would be expected if the intervention were contributing to producing the observed changes.

2. Compare the results to the counterfactual

Develop an estimate of what would have happened without the intervention and compare that to the findings of what happened with the intervention.

3. Investigate possible alternative explanations

Identify other factors that might have caused the impacts and see if it is possible to rule them out.

Resources

Recorded webinar: Jane Davidson's overview of options for causal inference in a 20 minute webinar in the American Evaluation Association's Coffee Break series.  Free to all, including non-members. 

Comments

Anonymous's picture
Cassie Bell

Hi. I work for a public sector union in Canada that is halfway through implementing a five-year education plan. While formative evaluation has been undertaken by staff educators and their supervisors, the summative evaluation is only being developped now. What would be the most effective evaluation to use to determine impact and/or outcomes of the program? 

Thanks! 

Patricia Rogers's picture
Patricia Rogers

Hi Cassie,

The most effective evaluation design or approach will depend on the nature of the intervention, the purposes of the evaluation and the availability of resources, especially time, money and existing data.  I suggest you check out our page on impact evaluation which provides guidance for working through the various issues you need to consider and options in terms of how you might go about it. https://www.betterevaluation.org/en/themes/impact_evaluation

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.