Investigate possible alternative explanations

All impact evaluations should include some attention to identifying and (if possible) ruling out alternative explanations for the impacts that have been observed.

Methods

  • Force field analysis

    A force field analysis is used to support the decision making process by providing a detailed overview of the variety of forces that may be acting on an organisational change issue. 

  • General Elimination Methodology

    General Elimination Methodology has two stages:

  • Key informant

    Asking experts of programmes or in the community to predict what would have happened in the absence of the intervention.

  • Multiple lines and levels of evidence

    Multiple lines and levels of evidence (MLLE) is a systematic approach to causal inference that involves bringing together different types of evidence (lines of evidence) and considering the strength of the evidence in terms of different indicators of a

  • Process tracing

    Process tracing is a case-based and theory-driven method for causal inference that applies specific types of tests to assess the strength of evidence for concluding that an intervention has contributed to changes that have been observed or measured.

  • RAPID outcomes assessment

    RAPID outcome assessment (ROA) is a method to assess and map the contribution of a project’s actions on a particular change in policy or the policy environment.

  • Ruling out technical explanations

    Ruling out technical explanations involves identifying and investigating possible ways that the results might reflect technical limitations rather than actual causal relationships.

  • Searching for disconfirming evidence/following up exceptions

    Treating data that doesn’t fit the expected pattern not as outliers but as potential clues to other causal factors and then seeking to explain them.

  • Statistically controlling for extraneous variables

    Statistically controlling for extraneous variables is an option for removing the influence of a variable on the study of program results.

Approaches

  • These approaches combine ruling out possible alternative explanations with options to check the results support causal attribution.

  • Contribution analysis

    Contribution analysis is an evaluation approach that provides a systematic way of understanding an intervention's contribution to observed outcomes or impacts.

  • Collaborative outcomes reporting

    Collaborative outcomes reporting (COR) is a participatory approach to impact evaluation based around a performance story that presents evidence of how a program has contributed to outcomes and impacts, that is then reviewed by both technical experts and

Expand to view all resources related to 'Investigate possible alternative explanations'

'Investigate possible alternative explanations' is referenced in:

Back to top