Investigate possible alternative explanations

All impact evaluations should include some attention to identifying and (if possible) ruling out alternative explanations for the impacts that have been observed.

Methods

  • Force field analysis

    A force field analysis is used to support the decision making process by providing a detailed overview of the variety of forces that may be acting on an organisational change issue. 

  • General Elimination Methodology

    General Elimination Methodology has two stages:

  • Key informant

    Asking experts of programmes or in the community to predict what would have happened in the absence of the intervention.

  • Multiple lines and levels of evidence

    Multiple lines and levels of evidence (MLLE) is a systematic approach to causal inference that involves bringing together different types of evidence (lines of evidence) and considering the strength of the evidence in terms of different indicators of a

  • Process tracing

    Process tracing is a case-based approach to causal inference which focuses on the use of clues within a case (causal-process observations, CPOs) to adjudicate between alternative possible explanations. 

  • RAPID outcomes assessment

    RAPID outcome assessment (ROA) is a method to assess and map the contribution of a project’s actions on a particular change in policy or the policy environment.

  • Ruling out technical explanations

    Ruling out technical explanations involves identifying and investigating possible ways that the results might reflect technical limitations rather than actual causal relationships.

  • Searching for disconfirming evidence/following up exceptions

    Treating data that doesn’t fit the expected pattern not as outliers but as potential clues to other causal factors and then seeking to explain them.

  • Statistically controlling for extraneous variables

    Statistically controlling for extraneous variables is an option for removing the influence of a variable on the study of program results.

Approaches

  • These approaches combine ruling out possible alternative explanations with options to check the results support causal attribution.

  • Contribution analysis

    Contribution Analysis is an approach for assessing causal questions and inferring causality in real-life program evaluations.

  • Collaborative outcomes reporting

    Collaborative outcomes reporting (COR) is a participatory approach to impact evaluation based around a performance story that presents evidence of how a program has contributed to outcomes and impacts, that is then reviewed by both technical experts and

'Investigate possible alternative explanations' is referenced in:

Back to top