Making a difference: M&E of policy research

The paper presents examples and approaches on conducting M&E of policy research from the current experience of a range of research institutes, think tanks and funding bodies.

The approaches have been divided into the following five key performance areas: Strategy and direction; Management; Outputs; Uptake; and Outcomes and impacts. Research programmes or institutes may wish to focus on only one of these areas, or may combine approaches across the areas to form a more comprehensive M&E plan.

Excerpt

"This paper aims to advance understanding on how to monitor and evaluate policy research; that is, research that is undertaken in order to inform and influence public policy. Policy is defined very broadly to encompass both policy decisions and processes, including implementation.The paper is written with research programmes and institutions in mind, rather than individual researchers." Hovland, I. (2007) p. vii

Contents

  • Acknowledgements
  • Acronyms and abbreviations
  • Executive summary
  • Introduction: The non-academic impact of research
  • Lessons from current experience: How to evaluate five key performance areas of policy research projects, programmes, and institutions
  • Institutional evaluations: Additional concerns
  • Conclusion: Designing and M&E plan for a policy research project, programme or institution
  • References
  • Appendix 1: Examples of institutional evaluations of policy research institutes and think tank.

Specific methods mentioned

  • Figure 1. Example of a Problem Tree p. 5
  • Figure 2. Stakeholder Analysis Grid  p. 5
  • Figure 3. Outline of a Logframe p. 6
  • Figure 4. An example of a SNA map p. 7
  • Figure 5. The process of constructing Impact Pathways and Narrative p. 9
  • Figure 6. Example of Problem Tree from the CPWF: Water and People in Catchments p. 10
  • Figure 6 (continued). Example of Objective Tree from the CPWF: Water and People in Catchments p. 11
  • Figure 7. Example of a Gantt chart p. 12
  • Figure 8. In the process of constructing a Network Map for one of the CPWF projects p. 13
  • Figure 9. Example of an Outputs x Impacts matrix p. 15
  • Figure 10. Example of an Outputs x Stakeholder matrix p. 15
  • Figure 11. The Appreciative Inquiry ‘5-D’ model p. 19
  • Figure 12. After Action Review questions p. 25
  • Figure 13. The three stages of Outcome Mapping p. 28
  • Figure 14. Example of ROA Output p. 30
  • Figure 15. Example of an Actor x Actor matrix p. 34
  • Figure 16. The CAF model p. 36
  • Figure 17. The POET model p. 37
  • Figure 18. IISD Policy Influence Report Card p. 54

Sources

Hovland, I. (2007). Making a difference: M&E of policy research, ODI Working Paper 281. London: Overseas Development Institute. https://odi.org/en/publications/making-a-difference-me-of-policy-research/

'Making a difference: M&E of policy research' is referenced in: