The working paper, written by Elliot Stern, Nicoletta Stame, John Mayne, Kim Forss, Rick Davies, Barbara Befani for the UK Department for International Development (DFID), describes how Theory based, Case-based and Participatory options can be used in impact evaluations. These designs show promise to reinforce existing IE practice, including Experimental and Statistical designs, when dealing with complex programmes.
For those with limited time, the Executive Summary provides an overview in general terms and each of the core chapters has a ‘Main Messages’ box at the beginning that highlights the key points in that chapter. The last chapter ‘Conclusions and Next Steps’ draws together 10 study conclusions.
Usefully, the report identifies not only design approaches but also variants of these approaches (or specific options). These are listed below:
- Causal process designs: Theory of Change, Process tracing, Contribution Analysis, Impact pathways
- Causal mechanism designs: Realist evaluation, Congruence analysis
- Interpretative: Naturalistic, Grounded theory, Ethnography
- Structured: Configurations, QCA, Within-Case- Analysis, Simulations and Network analysis
- Normative designs: Participatory or democratic evaluation, Empowerment evaluation
- Agency designs: Learning by doing, Policy dialogue, Collaborative, Action Research
The title of each chapter is presented below. The most important chapters, from a options perspective, are chapters 3, 4 and 5. Chapter 3 identifies and introduces the three main designs that show promise to reinforce existing IE practice when dealing with complex programmes – theory-based; case-based and participatory. How these designs are used in practice is further elaborated in Chapters 4 and 5.
- Chapter 1: Introducing the study
- Chapter 2: Defining Impact Evaluation
- Chapter 3: Choosing Designs and Options
- Chapter 4: Evaluation Questions and Evaluation Designs
- Chapter 5: Programme Attributes and Designs
- Chapter 6: Quality Assurance
- Chapter 7: Conclusions and Next Steps
Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. Department for International Development, (2012). Broadening the range of designs and options for impact evaluations (Working Paper 38). Retrieved from website: http://www.dfid.gov.uk/Documents/publications1/design-option-impact-eval.pdf