Guidelines for applying the climate and ecosystems health criterion in the commissioning, design and implementation of evaluations

This guide sets out the rationale for why climate and ecosystem health need to be addressed by all evaluations and how this might be done during the commissioning, design and conduct of an evaluation.

Authors and their affiliation

SAMEA (the South African Monitoring and Evaluation Association) working with the national Department of Planning, Monitoring and Evaluation (DPME)

Key features

A 25-page guide to addressing environmental sustainability in an evaluation in terms of climate and ecosystems health (CEH). The guideline sets out the rationale for why climate and ecosystem health need to be addressed by all evaluations and how this might be done during the commissioning, design and conduct of an evaluation, with examples from evaluations.

Contents

PART A: BACKGROUND TO THE GUIDELINE AND CRITERION FOR CLIMATE AND ECOSYSTEMS HEALTH

1. Introduction 1
1.1 Background 1
1.2 Purpose, objectives and structure of the guideline 2
1.3 Other guidelines 2
2. An overview: Climate and ecosystems health 3
2.1 The climate crisis 3
2.2 Ecosystems breakdown and biodiversity 4
2.3 The link between CEH and socio-economic justice and equity 4
2.4 Approaches to improving climate and ecosystems health 5
3. The climate and ecosystems health evaluation criterion: Definitions and dimensions 7
3.1 Focus and definition of the criterion 7
3.2 Dimensions of the CEH criterion 8
3.2.1 Dimension 1: Impacts of the intervention on CEH 8
3.2.2 Dimension 2: Impacts of the climate crisis and ecosystem breakdown on the intervention 9
4 CEH-related principles guiding the commissioning and undertaking of evaluations 10

PART B: INTEGRATING CEH INTO EVALUATIONS: COMMISSIONING, DESIGNING AND UNDERTAKING EVALUATIONS

5 Incorporating CEH into the evaluation process in a way that encourages the likelihood of use of the evaluation 11
6 Preparing for the commissioning of the evaluation 12
6.1 Deciding on how far to include CEH in the evaluation, and with what focus 12
6.2 Addressing capacity and expertise requirements 12
4.3 Encouraging stakeholder ownership and use of the CEH data and findings 14
7 Developing the terms of reference (ToR) 13
7.1 Evaluation purpose 13
7.2 Evaluation questions 14
7.3 Identifying evaluation users/stakeholders 14
7.4 Scope of the CEH application in the evaluation 16
7.5 CEH implications for evaluation design 16
7.6 Methodology: Types of data and data collection methods 17
7.7 Evaluation team 18
8 Managing the evaluation 18
8.1 Steering Committee 18
8.2 Peer reviewers 18
8.3 The role of sponsors/funding partners 25
8.4 Minimising resource consumption and waste/pollution when undertaking the evaluation 19
8.5 Validation of findings 19
8.6 Quality assessment of the evaluation 19
9 Bringing a CEH lens into the follow-up to the evaluation 19
9.1 Improvement plan and progress report 19
9.2 Communicating the results of the evaluation 19

ANNEXES 20

Annex 1: Sources 20
Annex 2: South African government CEH-related policies 21
Annex 3: Considerations in identifying relevant stakeholder groups 22
Annex 4: Case studies: Applying the CEH criterion to evaluation TOR 23

How have you used or intend on using this resource?

I want to refer back to the section explaining why this is needed, to help make the argument. I also intend to use it to guide the design and reporting of evaluations which include environmental sustainability. 

Why would you recommend it to other people?

While this is intended to be directly relevant to evaluators and evaluation commissioners working in South Africa, it also has wider relevance to people working in other countries. The list of relevant national policies and international commitments is likely to be particularly useful, as are the three case studies showing how an evaluation Terms of Reference could be revised to incorporate environmental sustainability. The explanation of why this is important could also be useful for advocating for environmental sustainability to be included in all evaluations.