Compare results to the counterfactual

One of the three tasks involved in understanding causes is to compare the observed results to those you would expect if the intervention had not been implemented - this is known as the 'counterfactual'.

Many discussions of impact evaluation argue that it is essential to include a counterfactual.  Some people however argue that in turbulent, complex situations, it can be impossible to develop an accurate estimate of what would have happened in the absence of an intervention, since this absence would have affected the situation in ways that cannot be predicted. In situations of rapid and unpredictable change, when it might not be possible to construct a credible counterfactual it might be possible to build a strong, empirical case that an intervention produced certain impacts, but not to be sure about what would have happened if the intervention had not been implemented.

For example, it might be possible to show that the development of community infrastructure for raising fish for consumption and sale was directly due to a local project, without being able to confidently state that this would not have happened in the absence of the project (perhaps through an alternative project being implemented by another organization). 

For a discussion about counterfactual approaches to causal inference, see The Stanford Encyclopedia of Philosophy entry.

Methods

There are three clusters of methods for this task:

Experimental methods (or research designs)

Develop a counterfactual using a control group. Randomly assign participants to either receive the intervention or to be in a control group.

Quasi-experimental methods (or research designs)

Develop a counterfactual using a comparison group which has not been created by randomization.

Non-experimental methods

Develop a hypothetical prediction of what would have happened in the absence of the intervention.

Approaches

Expand to view all resources related to 'Compare results to the counterfactual'

'Compare results to the counterfactual' is referenced in: