C4D Hub: Investigate possible alternative explanations (strategy 3)

What is it?

This strategy involves looking at the evidence at hand, and systematically identifying other possible causes of changes (such as other programs, external political and social changes etc.), and then investigating the extent to which they contributed the change. Often there are multiple causes for any given change, so this process is also about understanding the relative contribution of multiple factors. It is useful to a strong program theory or logic model, and use this to think about alternative explanations at each level of the change theory. This strategy is best used in combination with strategies to check the consistency of evidence, and in this way offers a way to engage in credible causal analysis without a counterfactual. 

General information

There are many options for identifying and ruling out other possible explanations. A full list with details and resources is available on the Better Evaluation website. This page is recommended reading before considering options to apply to C4D.

Ruling out possible alternative explanations and C4D

Applying the C4D principles

Complex 

Using this strategy is important in complicated and complex situations. It is almost a given in C4D that social and behavioural changes will have multiple causes, some of which may be predictable, and others which won't be. Because of this, this strategy may not so much be about 'ruling out' alternative explanations, but instead making a judgement about the extent of the contribution of C4D in the context of identified multiple causes.  

Holistic 

Some openness to challenging and problematising assumptions and being surprised by findings is important when undertaking this task. 

Participatory 

A participatory approach to draw on a range of different perspectives and knowledge would strengthen the findings from this task.  

 

Realistic 

There are many practical and feasible options for investigating alternative explanations. Even very models R,M&E Frameworks and studies could include these options to greatly improve the ability to make clear, evidence-based causal inference.  

Recommended options and adaptations for C4D

(A combination of strategies is usually advisable)

General Elimination Methodology: a process of identifying alternative explanations and then systematically investigating them to see if they can be ruled out. A range of different, open-ended methods can be used to investigate alternative explanations. It is consistent with the C4D Evaluation Framework in the following ways:

  • realisticthe methods for elimination and investigation should be flexible and pragmatic
  • holisticthe process ensures a holistic understanding of changes and contexts, and not just cherry-picking evidence that supports the theories
  • participatorywith flexibility and creativity, this process could be adapted to include stakeholders and communities in brainstorming alternative explanations and investigations.

 

Key informant: key informant interviews with experts and community members to identify possible explanations for change, and to assess whether these explanations can be ruled out. It is consistent with the C4D Evaluation Framework in the following ways:

  • holisticthe process ensures a holistic understanding of changes and contexts, and not just cherry-picking evidence that supports the theories
  • participatorythis process ensures that a range of different perspectives are included. 

Process tracing: going through each stage of the theory of change and considering whether there are plausible alternative explanations at each step. It is consistent with the C4D Evaluation Framework in the following ways:

  • realisticthis is a relatively simple and pragmatic process that can easily be added to a combination of other options.
  • participatory: with flexibility and creativity, this process could be adapted to include stakeholders and communities.

Searching for disconfirming evidence/Following up exceptions: There are usually outlying cases in any data, that stand out as not following the pattern (both positive outliers and negative outliers). These exceptions can give potential clues about causal factors. Similarly, further investigations into cases that seem to disconfirm the theory and trying to explain can yield important insights about causal mechanisms and contexts. It is consistent with the C4D Evaluation Framework in the following ways:  

  • complexthis process is consistent with complexity theories since it recognises that the same factors and conditions affect different people in different ways, and seeks to use that to learn and adapt.

Resources

Contribution analysis is a process that combines processes to Check the results support causal attribution (strategy 2) and strategies to investigate possible alternative explanations (strategy 3). For a resource detailing how to undertake contribution analysis click here.

Examples

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.