The evaluation of politics and the politics of evaluation

This paper, written by Chris Roche and Linda Kelly for the Development Leadership Program (DLP), argues for the use of a mixed methods approach to monitoring and evaluation of programs that 'work politically'.

It outlines guidance on evaluating, working and thinking politically and provides detailed examples of different methods that can be used when assessing these kinds of programs.

Excerpt

"The aim of this paper is to contribute to the argument for a ‘mixed methods’ approach to monitoring and evaluation, particularly for programs that seek to ‘work politically’. We suggest that this is particularly relevant where the object of evaluation is a process or a practice that is intended to change power relations, rather than the more easily quantifiable measurement of the delivery of a service (such as clean water) or specific items (such as kilometres of road or bed-nets). We argue that different policy, program and operational objectives require different methods of evaluation and, often, mixed or combined ones, depending on the range of objectives and the intended sequence of outcomes. The paper outlines a range of these methods and illustrates their usefulness for the evaluation of different processes and interventions. It also suggests that part of the explanation for the resistance by parts of the development community to some forms of evaluation, as discussed in this paper, can be attributed to the political imperatives of donors and other development organizations." (Roche & Kelly, 2012)

Contents

• Context 4
• Evaluating working and thinking politically 6
• Different approaches to the evaluation of ‘politics’ 8
  • Network mapping 12
  • Outcome mapping 14
  • Randomised Controlled Trials 17
  • Quasi-experimental approaches 19
  • ‘Most Significant Change’ and ‘Narrative Analysis’ 21
  • Participatory evaluation 23
  • Assessing cost effectiveness and value for money 24
  • ‘Action-Reflection’ and ‘Action Research’ approaches 28
• And the politics of evaluation? 34