Evaluations that make a difference: Stories from around the world

This week sees the launch of the Evaluations that make a difference blog series, which will release a story about an evaluation which has made a difference to people's lives, every week starting today. This project, which was supported through an EvalPartners-Innovation Challenge grant, aimed to strengthen the demand for and use of evaluation to inform policy making during the International Year of Evaluation (2015). In this guest blog, Martha McGuire gives an overview of the process behind the project and how you can get involved.

What is the value of evaluation?  What factors contribute to making an evaluation more or less valuable? What are the benefits and how are they expressed or measured? How can they be described so that they make sense to citizens and to policy makers? Can stories provide a good way of communicating evaluation findings?

A group of evaluators from around the world received an EvalPartners Innovation Grant to collect stories that would help answer these questions. 

The result was Evaluations that make a difference - a collection of eight evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives.  This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users.  The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs. 

The following group served as the editorial board for the project:

• Burt Perrin and Ramon Crespo– European Evaluation Society (EES)
• Rochelle Zorzi – Canadian Evaluation Society (CES)
• Pablo Rodriguez-Bilella – Red de Seguimiento, Evaluación y Sistematización en América Latina y el Caribe (ReLAC)
• Scott Bayley – Australasian Evaluation Society (AES)
• Serge Eric Yakeu – African Evaluation Association (AfrEA)
• Soma De Silva – Sri Lanka Evaluation Association (SLEvA)

The first step was to define what was meant by evaluations that make a difference. Many evaluations have sound methodologies and, in some cases, get used to inform decisions and improve programs, organizations and policies (evaluation influence). However, this exploration went beyond that and looked at what evaluations can do to improve people’s lives (evaluation impact) and contributed to social betterment.

The process

Eight ways to enhance evaluation impact:

In addition to the eight stories about evaluations that have made a difference in people’s lives, this project teased out the “enabling factors” that contributed to their impact, highlighted for the benefit of evaluators and evaluation users who want to do more impactful evaluations. 

1. Give voice to the voiceless – all voices need to be heard, but those without power often go unheard

2. Provide credible evidence based on excellent design and methodologies

3. Use a positive approach emphasizing strengths, opportunities, aspirations and results

4. Actively engage users & intended beneficiaries through a utilization-focused process that gets buy-in as the evaluation progresses

5. Embed evaluation within the programme right from the start, if possible, in order to have base-line information and to promote evaluative thinking

6. Sincerely care about the evaluation so that commissioners, users and evaluators are working together to ensure credible evidence

7. Champion the evaluation – evaluators need to work with commissioners and users to help them understand how the evaluation can contribute to making decisions

8. Focus on evaluation impact – from the beginning evaluators need to think about the potential effect of the evaluation on the program and program participants.

Join the discussion

The full collection of stories can be accessed online, however those of us involved in the analysis of the stories concluded that more research and discussion is needed. The stories provided some excellent insights into the factors that contribute to evaluations making a difference. We would now like to explore each of the factors more deeply.

To this end, every 10 days, beginning April 11, 2015, we will upload a story to the EvalStories Blog at: Evaluation Stories (French and English) and AlBorde del Caos (Spanish). Readers are encouraged to engage with these blogs by joining the discussion and sharing your experiences. We would like to hear more stories. 

Blog schedule

How an evaluation led to rapid change: Salvaging Sri Lanka’s small and medium businesses

April 11, 2016 – a story from Sri Lanka - Read this story

Tumekataa kula mavi tena! We refuse to eat shit!

April 21, 2016 – a story from Kenya

Evaluation in action:  the Milne Bay emergency phone service

May 2, 2016 – a story from Papua New Guinea

Positive sisters: a transformative journey

May 13, 2016  - a story from the Netherlands

Learning and earning: training that works

May 23, 2016 – a story from Kenya

The power of community owned data

June 3, 2016 – a story from Canada

If you don’t ask, you won’t see it

June 13, 2016 – a story from Mexico

Listening to the listeners

June 23, 2016 – a story from Nepal

Image: “Tales of a Grandfather”, Albert Anker (1884)