Process Tracing and Contribution Analysis: A Combined Approach to Generative Causal Inference for Impact Evaluation

This article, written by Barbara Befani and John Mayne for the IDS Bulletin (Volume 45 Number 6), outlines how the combined use of contribution analysis (CA) with process tracing (PT) can shift the focus of impact evaluation from ‘assessing impact’ to ‘assessing confidence’ (about impact). The paper uses a case study of the evaluation of the contribution of a teaching programme to the improvement of school performance of girls to demonstrate that the combination of CA and PT is preferable to using them on their own.

Excerpt

"It is becoming increasingly seen that for exploring causality in many real-world intervention settings, alternatives to the traditional counterfactual approaches need to be used, such as discussed in Stern et al.(2012) and Mayne and Stern (2013). Among the reasons for this are that many evaluations are conducted after the intervention is in place; ethical considerations limit the use of random assignment; interventions may be aimed at the entire population; or, with the increasing complexity of many interventions, setting up counterfactuals is not possible or practical."

Contents

  • Introduction
    • Perspectives on causality
      • Generative causality in contribution analysis
    • Contribution analysis
    • Process tracing
      • Using evidence in PT
      • The different implications of confirmatory evidence
  • Carrying out contribution analysis applying the tests and principles of process tracing
    • Setting up the case
      • The problem, as seen at the time the intervention started
      • The intervention
      • The overall ToC
      • Prior evidence for the intervention
      • Main evaluation findings
      • A ToC for the intervention
    • The analysis
      • Maximising certainty: ruling out possible causal factors through hoop tests
        • Assessing the contribution of the intervention to improvement of girls’ school performance (PT1)
        • Assessing the relevance of other factors (PT2)
        • Testing the ToC as a whole (PT3)
  • Concluding remarks: from ‘assessing impact’ to ‘assessing our confidence’ about impact

Sources

Befani, B. and Mayne, J. (2014). 'Process Tracing and Contribution Analysis: A Combined Approach to Generative Causal Inference for Impact Evaluation.' IDS Bulletin, Special Issue: Rethinking Impact Evaluation for Development by Barbara Befani, Chris Barnett and Elliot Stern. Volume 45, Issue 6, pages 17–36. Retrieved from: http://onlinelibrary.wiley.com/doi/10.1111/1759-5436.12110/abstract