What can we learn from evaluation advocates to help embed evaluation in organisations?

Integrating evaluation into the routine operations of an organisation isn’t easy. In addition to policies and training, it requires the energy and commitment of individuals who help keep evaluation on the agenda. This blog shares insights from Alison Rogers’ doctoral thesis research with evaluation advocates working to embed evaluation in Australian non-profit organisations. The research involved interviewing 17 participants and conducting four organisational case studies. The researchers used social interdependence theory (Johnson, 2003) to interpret the strategies participants used to embed evaluation and found participants used elements of cooperative teamwork.

Evaluation advocates work within organisations, often outside formal evaluation roles, to help enable and sustain evaluation as a day-to-day activity. They motivate others and bring energy, interest, and enthusiasm to help overcome structural barriers, resource constraints, and interpersonal challenges that can arise when efforts are made to embed evaluation activities within an organisation. Rather than focusing only on systems and tools, they connect evaluation to colleagues’ aspirations and organisational goals, helping them to make judgements about effectiveness and quality that support learning and inform decision making.

Evaluation advocates can be hard to identify because they often work behind the scenes. Our initial research generated a description to help make them visible (Rogers & Gullickson, 2018). The case studies then involved interviewing the colleagues of evaluation advocates to help us understand the types of knowledge, strategies, and attributes these advocates displayed in their work. From the advocates and their colleagues in our sample we learnt that people in this role:

• Are curious lifelong learners, generalists and critical thinkers who seek out expertise and learn from it
• Adapt information, tools and new knowledge for different audiences
• Harness their networks, listen and communicate effectively, and model desired behaviours to enable change

Advocates in our study had attributes that help them achieve results. They were positive, enthusiastic, passionate about evaluation. They were willing to take risks, make mistakes and drive the process toward their vision of integrated evaluation. When things didn’t go to plan, they were persistent, tenacious, resilient, and patient.

Through our research, we identified a number of strategies that advocates used for embedding and sustaining evaluation within organisational systems and practice. While drawn from the not-for-profit sector, these strategies are likely useful for a range of organisational settings, regardless of the type of organisation, as they primarily focus on interpersonal strategies when working with teams.

The strategies shared by participants reflect five elements of cooperative teamwork derived from a social psychological theory (Johnson, 2003):

• Positive interdependence: Establishing cooperative relationships with the understanding that everyone needs each other for success
• Individual accountability: Doing one’s fair share
• Social skills: Communicating, resolving conflicts, and demonstrating cultural competence
• Group processing: Reflecting on how well the group is functioning.
• Promotive interaction: Providing encouragement

When asked how they supported the embedding of evaluation in organisational systems, all the evaluation advocates I spoke to, no matter their level on the organisational hierarchy, all shared multiple strategies that aligned with each of the following three elements: positive interdependence, promotive interaction, and social skills. These strategies included:

Positive interdependence (finding shared goals):

• Considering how evaluation can be strategically promoted and used for organisational change
• Advocating for support and resources
• Developing engaging ways to explain details and build shared vision

Examples included creating mind maps, charts or logic diagrams to connect an individual’s work plan to evaluation activities and organisational goals, so the team member can see how their evaluative work contributes to achieving the overall goal.

Promotive interaction (providing encouragement):

• Motivating others and bring energy, interest, and enthusiasm
• Supporting colleagues to apply evaluative thinking, use findings, and reflect on practice

For example, recognising and celebrating colleagues’ efforts and achievements around evaluation in order to help team members feel encouraged and appreciated, and to help build momentum for evaluation.

Social skills (being inclusive):

• Assisting, training, mentoring, and encouraging inclusive participation in evaluation
• Creating space for critical questions and open discussion
• Sharing tools, resources, networks, and access to expertise

Advocates ensured communication around evaluation was clear, timely, appropriate, and accessible so that team members understand what was happening and why. For example, developing ways of sharing information collaboratively resulted in using analogies, regular items on evaluation within existing meetings, workplace posters and distributing evaluation materials on internal organisational social media platforms.

Evaluation advocates in senior or influential positions also described using strategies aligned with individual accountability and group processing. Their organisational positions enabled them to influence systems and processes in ways that made evaluation more routine, embedded, and sustainable. This included supporting structural changes, strengthening internal capacity, facilitating reflections on group dynamics, and holding individuals accountable.

Individual accountability means ensuring that people are held responsible for their contribution to the team work and do what they say they are going to do. Participants mentioned a range of strategies to ensure that responsibility for evaluation was shared, visible, and supported across roles. These included:

• Providing training, instruction manuals, short guides, and peer support mechanisms to provide staff with the information and support they needed to engage with evaluation tasks
• Encouraging others to adapt tools for their own purposes and integrate them as essential components of project design
• Embedding evaluation tasks alongside other core business tasks that include accountability mechanisms to signal that evaluation is part of everyday work, not an optional add-on.
• Sharing responsibility for monitoring and evaluation across roles, documenting responsibilities, and creating mutual agreement about appropriate accountability mechanisms
• Incorporating accountability into existing work plans, to ensure evaluation responsibilities were not seen as optional or extra, but part of everyday work
• Formally transferring ownership of evaluation responsibilities to other roles as needed
• Allocating responsibility among staff to publish findings in multiple formats and disseminate them widely to demonstrate how useful evaluation findings can be for a variety of key audiences such as decision makers who then may demand more and elevate evaluation as a priority  
• Developing processes to hold staff within the team and across organisations mutually accountable to foster shared measurement opportunities.
• Recruiting staff with an aptitude for evaluation, fostering the development of this skill in individual’s role and incorporating this into professional development plans

In practice, this could involve using calendars, tasking tools, spreadsheets or databases to designate responsibilities to individuals or teams. It is about using tools to ensure that team members understand their responsibilities and have visibility over how other people are also doing their fair share.

Group processing focuses on how the team is functioning and reflecting on the group dynamics. It involves embedding evaluative reflection to understand more about team practices, and creating opportunities for ongoing learning and collaboration. Strategies included:

• Incorporating evaluative questions into routine meetings and general conversations to normalise judgement and reflection on teamwork dynamics
• Using prompts to encourage reflection about core business, evaluation processes, and team dynamics in organisational systems, helping to institutionalise reflection at all levels
• Collaborating and involving people in evaluation processes from the outset, to foster two-way feedback mechanisms and encourage further engagement in evaluation
• Partnering with critical friends, academics, other organisations, and evaluators to seek feedback and obtain an external perspective on the way the team is working
• Networking, forming alliances, and talking about evaluation with others to build coalitions of support, generate demand, and seek feedback on relationships and teamwork
• Engaging funders in shared learning about what evidence is available and worth collecting, and negotiating agreed indicators to include in formal agreements

In essence, these strategies are about team members taking time for reflection. It may involve instigating a process at the outset for disseminating achievements, airing issues, and dealing with grievances.

The strategies associated with individual accountability and group processing extended the influence of evaluation advocates by embedding evaluation more deeply into organisational systems and culture.

When the elements of the cooperative teamwork are intentionally incorporated into teams working on evaluation projects, they are more likely to result in initiatives that are sustainable. Evaluation advocates in higher-levels or positions of influence tended to work intentionally and incorporate strategies that aligned with all five elements. For those in less senior roles, focusing on interpersonal and motivational strategies and nuanced support was particularly useful. Incorporating strategies that align with individual accountability and group processing, to make their efforts less person-dependent, could support individuals to amplify their impact and further harness their contribution.

It can be challenging to make evaluation relevant, meaningful, and useful for non-evaluators working in organisations. Understanding more about evaluation advocates provides one pathway to help organisations meet the multiple demands for information. They are people who can effectively influence their co-workers to engage with evaluation and increase the uptake. This research may enable evaluators and decision makers to engage with non-evaluators and learn about how they contribute to building a culture of evaluative inquiry and increase the likelihood of embedding evaluation in organisational systems.

You can find further tools and insights on the BetterEvaluation method page, including:

• A field guide to identifying evaluation advocates
• Real-world examples of these strategies in action
• Information about the research design and references

We hope you will use these resources and strategies to stimulate ideas for your own practice and, with your communities of practice, to reflect on how these strategies align with you real-world experiences and how they might be adapted in your own organisational context.