C4D: Investigate causal attribution and contribution

What is it?

For most evaluations it is not enough to just gather and report data about activities and changes in conditions (expected results) - there needs to be an investigation of the role of the intervention in producing these results. This is needed for any outcome or impact evaluation and also for any evaluation that examines effectiveness or ways to improve performance.

In evaluation, causal attribution and contribution refer to being able to be confident there is a causal link between events – in particular between activities and results. The term ‘causal attribution’ refers to a direct causal link. The term ‘causal contribution’ can be used to recognise multiple contributing factors that produce results. The term ‘causal inference’ covers both of these.

There are three main strategies for exploring causal inference. These are outlined below. This video provides an overview of the three main strategies.

Communication for Development (C4D) : 

Compares the observed results to an estimate of what have been the situation if the intervention had not been implemented, often by creating or identifying a group of similar people who have not participated in a program.

Examines whether the data are consistent with the theory of change – in particular seeking out data that doesn’t match (for example the timing of the change makes it not plausible that it was due to the intervention).

Identifies other possible explanations (for example, the activities of another program) and then investigates whether these can be ruled out.

General information

A UNICEF Office of Research Methodological Brief on Strategies for Causal Attribution (by Patricia Rogers) provides a good general overview of all three strategies. Another recommended general resource is Impact Evaluation: A Guide for Managers Elliot Stern. The Rainbow Framework's cluster of tasks on understanding causes also provides information on all three strategies. These resources are recommended background reading/viewing before considering methods that could be applied to C4D.

Applying the C4D principles

Holistic

When selecting from strategies consider:

  • Strategies to create a counterfactual (strategy 1) are often not suitable because they distort how the intervention might work in the 'real world' contexts. Strategies to check the results support causal attribution are more sensitive to context and interconnections.
  • Strategies for investigating possible alternative explanations (strategy 3) are important for challenging and problematising assumptions as part of a holistic approach.

Complexity

To understand the causal contribution it is important to also understand the contributions of other programs and contextual factors. Strategies to investigate this must be in the evaluation design.

Learning-based

The learning needs may determine which combination of strategies will be most useful. While designs creating a counterfactual (strategy 1) are best in situations where strong hypotheses (theories) are known and need to be tested and proven, they are not as well suited in more exploratory situations. A combination of Strategy 2: 'Check the results support causal attribution' and Strategy 3: 'Investigate possible alternative explanations' can be used where there is a need to learn about and better understand causes and changes.

Critical

It is important to pay attention to the different ways that C4D initiatives affect different groups. Counterfactual-based designs (strategy 1) can show differences experienced by different groups through data disaggregations (looking at different variables). However, mechanisms to create comparison groups (such as incentives) may disguise power differences. Critical reflection on power dynamics and inclusion might therefore make Strategy 2: Check the results support causal attribution and Strategy 3: Investigate possible alternative explanations better methods.

Accountable

A central question in RM&E from an accountability perspective is 'what has been the impact (or contribution) of C4D to observed changes'. Answering this question rigorously requires selecting carefully from three causal analysis strategies.

Realistic

Feasibility and availability of expertise might be factors when deciding on methods for investigating causes. Experimental and quasi-experimental designs (strategy 1) don’t necessarily take more time and resources, but they do depend on a number of practical factors including: upfront investment in planning and design; and the ability to plan the C4D intervention around the needs of the experiment. Where these things are not possible, it might be more pragmatic to use Strategy 2: Check the results support causal attribution and Strategy 3: Investigate possible alternative explanations (strategy 3).

Recommended methods and adaptations for C4D

Casual Contribution and C4D

  • In C4D it is often more useful to think about investigating 'causal contribution', rather than 'causal attribution'. Thinking in term causal contribution recognises that multiple factors contribute to changes. In UNICEF, for example, C4D and Program teams are often interested in investigating the contribution of C4D components of programs to the outcomes and impacts that are observed.

Selecting Strategies for Investigating Causal Contribution in C4D

  • There are three broad strategies for answering causal questions and C4D R,M&E might use a combination of these.

    Communication for Development (C4D) : 

    An estimate of what would have happened in the absence of a program.

    While designs that include counterfactual are considered by some to be the 'gold standard', for many C4D initiatives a credible counterfactual will not be possible. This is especially the case in programs where participants are volunteers or are specially selected for participation, and for national level programs. In these cases you will need to use the other two strategies (often in combination) for causal inference.

    If you don't have, or can't create, a credible counterfactual...

    Look systematically at whether the evidence is consistent with what would be expected if the intervention was producing the observed changes.

    Identify possible alternative explanations and investigate whether these can be ruled out.

In this section

Expand to view all resources related to 'C4D: Investigate causal attribution and contribution'

'C4D: Investigate causal attribution and contribution' is referenced in: