Week 38: Addressing sustainability in evaluation
This year’s AEA theme of visionary evaluation challenges evaluators to consider how their work can contribute to a sustainable future. In this week’s blog post, we’ve asked Will Allen for advice about how evaluations can address sustainability.
When we talk about a “sustainable future” what do we mean?
Since the 1970s, discussions about a sustainable future have focused on environmental issues. On a global level around issues such as the ozone layer, climate change, biodiversity, ocean acidification. At a more local level, communities are concerned about their catchments and watersheds, water quality, and land degradation. In companies and organizations, people think about energy efficiency or recycling. In more recent years, there are a couple of other frames that have emerged and are starting to change the way we think about sustainability.
One is a growing appreciation that there is no one simple answer to these issues. It is often more about how different groups work together – because everyone has to play a part in addressing complex issues. So it’s not just about developing a solution for say, a natural resource management issue, it’s also about how different organisations, companies, farmers and other actors will work together to implement it. This is why today there is such an emphasis on participation and collaboration because without this it won’t be possible to create a sustainable future.
More recently, there has been a growing recognition that human well-being not only requires a global society to live within planetary environmental boundaries, it requires us to pay equal attention to ensure that all people have access to the resources they need to be able to meet their human rights.
Sustainability therefore is starting to be thought of as not only about the environment, but about human rights. These rights include ensuring people have access to food, water, health, security and that they have a voice in their decision making. From this perspective we need to place a real focus on those people whose voice is often missed out or marginalized – taking particular care to ensure the input of societal sectors such as Indigenous peoples, and women. In a complex world we are not just looking at setting one path to the future, but working inclusively with a range of stakeholders to ensure a diversity of development pathways to safeguard both the environment and human rights.
Some programs have sustainability as a stated goal, but there are still many that do not. Should evaluators try to address sustainability even when it’s not a stated goal?
Yes, they should, for a variety of reasons.
Firstly, because paying attention to the wider implications of sustainability an evaluation is likely to uncover benefits beyond the immediate program goals. In this way evaluation can send signals that the broader sense of sustainability is valued and help to build an organisation’s or agency’s awareness of this and their capacity to address it.
Secondly, doing so will most likely support program managers to tell a better story about their broader impact. Programs often do more than simply achieve their stated goals. In particular, many programs build team and community capacity that can be used for other purposes.
Thirdly, focusing on sustainability is an ethical imperative. If we go back to the American Evaluation Association’s principles, for example, number 5 states that evaluators have a responsibility for general welfare. Evaluators are encouraged to take a bigger picture of the impact of programs, policies and organisations.
So how can evaluators address sustainability in evaluations where it is not a formal, stated goal or principle?
In terms of practical actions, I’d suggest the following.
1. Think about sustainability as a process not a goal
Most importantly, we need to remember that sustainability is not an externally designed goal that we need to achieve. Indeed, it's better seen as a measure of the relationship between our communities as learners and our linked ecological, economic and social environments. Evaluation is not just about assessing the extent to which a particular sustainability goal has been achieved, or is likely to be achieved. It also requires us to support the abilities of individuals, organisations and communities to work together, to think about what they are trying to achieve, the paths to achieve this, and to learn and adapt during the journey.
2. Start evaluation earlier
One of the implications of this is that we need to ensure more evaluation exercises begin earlier in the process – into the planning stages. As evaluators we've got many tools and approaches that we can use almost more effectively at the beginning of a program, than by coming in with them at the end. Evaluative thinking, frameworks, and methods can help to clarify what programs are trying to do and how.
3. Stay involved through the program
Similarly, we need to look for opportunities to develop a role that enables us, as evaluators, to stay involved with the program or organisation along the way. This role is that of a critically reflective friend, supporting program participants with their ongoing learning and adaption. It asks evaluators to find ways to facilitate and help stakeholders to think more critically about their programs and their impacts, supporting innovation and adaptation.
4. Use a suite of tools, methods and approaches
It is an exciting time for evaluators to get more involved with sustainability programs because we can bring such a useful range of tools, methods and approaches to deal with the broader issues involved. And so most sustainability evaluations will utilise multiple tools and approaches. For example, systems thinking, theory of change, and outcomes-oriented evaluation approaches can be used to help program staff to address the complexity of sustainability. Frameworks for thinking about impacts across different social value areas include Triple (or Quadruple) Bottom Line and Value for Money. Outcome mapping and rubrics can be used to provide different ways to assess values, and help to clarify intermediate outcomes and indicators.
Of course we not only need to have a range of tools, but also good processes for using them. And today we see this sharing of experience not only happening through conferences, but also being made more readily available through e-forums such as BetterEvaluation and SEAChange.
Systems Concepts and Tools: Bob Williams’ site on systems thinking and evaluation.
Between social and planetary boundaries: navigating pathways in the safe and just space for humanity: Melissa Leach, Kate Raworth, Johan Rockstrom (2013) Between social and planetary boundaries: navigating pathways in the safe and just space for humanity, World Social Science Report 2013.