Evaluations that make a difference: Stories from around the world
What is the value of evaluation and can stories provide a good way of communicating evaluation findings?
In this guest blog post, Martha McGuire introduces the project, 'Evaluations that make a difference', a collection of eight evaluation stories from around the world about the ways in which evaluations contributed to making an impact, from the perspectives of the users and beneficiaries who have been involved in the evaluations.
What is the value of evaluation? What factors contribute to making an evaluation more or less valuable? What are the benefits and how are they expressed or measured? How can they be described so that they make sense to citizens and to policy makers? Can stories provide a good way of communicating evaluation findings?
A group of evaluators from around the world received an EvalPartners Innovation Grant to collect stories that would help answer these questions.
The result was Evaluations that make a difference - a collection of eight evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives. This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users. The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs.
The following group served as the editorial board for the project:
- Burt Perrin and Ramon Crespo– European Evaluation Society (EES)
- Rochelle Zorzi – Canadian Evaluation Society (CES)
- Pablo Rodriguez-Bilella – Red de Seguimiento, Evaluación y Sistematización en América Latina y el Caribe (ReLAC)
- Scott Bayley – Australasian Evaluation Society (AES)
- Serge Eric Yakeu – African Evaluation Association (AfrEA)
- Soma De Silva – Sri Lanka Evaluation Association (SLEvA)
The first step was to define what was meant by evaluations that make a difference. Many evaluations have sound methodologies and, in some cases, get used to inform decisions and improve programs, organizations and policies (evaluation influence). However, this exploration went beyond that and looked at what evaluations can do to improve people’s lives (evaluation impact) and contributed to social betterment.
The process
Eight ways to enhance evaluation impact:
In addition to the eight stories about evaluations that have made a difference in people’s lives, this project teased out the “enabling factors” that contributed to their impact, highlighted for the benefit of evaluators and evaluation users who want to do more impactful evaluations.
1. Listen to and amplify the voices of marginalised people – all voices need to be heard, but those without power often go unheard
2. Provide credible evidence based on excellent design and methodologies
3. Use a positive approach emphasizing strengths, opportunities, aspirations and results
4. Actively engage users & intended beneficiaries through a utilization-focused process that gets buy-in as the evaluation progresses
5. Embed evaluation within the programme right from the start, if possible, in order to have base-line information and to promote evaluative thinking
6. Sincerely care about the evaluation so that commissioners, users and evaluators are working together to ensure credible evidence.
7. Champion the evaluation – evaluators need to work with commissioners and users to help them understand how the evaluation can contribute to making decisions
8. Focus on evaluation impact – from the beginning evaluators need to think about the potential effect of the evaluation on the program and program participants.
Join the discussion
The full collection of stories can be accessed online, however those of us involved in the analysis of the stories concluded that more research and discussion is needed. The stories provided some excellent insights into the factors that contribute to evaluations making a difference. We would now like to explore each of the factors more deeply.
To this end, every 10 days, beginning April 11, 2015, we will upload a story to the EvalStories Blog at: Evaluation Stories (French and English) and AlBorde del Caos (Spanish). Readers are encouraged to engage with these blogs by joining the discussion and sharing your experiences. We would like to hear more stories.
Blog schedule
How an evaluation led to rapid change: Salvaging Sri Lanka’s small and medium businesses
April 11, 2016 – a story from Sri Lanka
Tumekataa kula mavi tena! We refuse to eat shit!
April 21, 2016 – a story from Kenya
Evaluation in action: the Milne Bay emergency phone service
May 2, 2016 – a story from Papua New Guinea
Positive sisters: a transformative journey
May 13, 2016 - a story from the Netherlands
Learning and earning: training that works
May 23, 2016 – a story from Kenya
The power of community-owned data
June 3, 2016 – a story from Canada
If you don’t ask, you won’t see it
June 13, 2016 – a story from Mexico
Listening to the listeners
June 23, 2016 – a story from Nepal
Image: “Tales of a Grandfather”, Albert Anker (1884)