What can we learn from qualitative impact evaluations about the effectiveness of lobby and advocacy? A meta-evaluation of Dutch aid programmes and assessment tool

This paper presents the results of a meta-evaluation that studied evaluations of lobby and advocacy (L&A) programs across Asia, Africa and Latin America.

The evaluations studied used "small-n qualitative approaches", which the authors note "are available to measure the effectiveness of L&A which use the theory of change as their analytical framework."

The researchers developed a tool to assess the confidence of the causal claims provided in the evaluations. Using this tool, the study found that predictable sources of bias were not addressed, and "assessments of the contribution of the programmes to the changes in outcomes were not provided in many of the evaluations."

The paper provides guidance on how to improve the design, conduct and reporting of small-n qualitative evaluations of L&A programs and of qualitative impact evaluations more broadly.