Integrating a transformative equity criterion into evaluations for promoting transformative systemic change

This guide sets out the rationale for why transformative equity needs to be addressed by all evaluations, especially in the South African context of high inequality, and how this might be done during the commissioning, design and conduct of an evaluation.

Authors and their affiliation

SAMEA (the South African Monitoring and Evaluation Association) working with the national Department of Planning, Monitoring and Evaluation (DPME)

Key features

A 39-page guide to addressing transformative equity in evaluation through a specific evaluation criterion. The guideline sets out the rationale for why transformative equity needs to be addressed by all evaluations, especially in the South African context of high inequality, and how this might be done during the commissioning, design and conduct of an evaluation, with examples from evaluations.

Transformative equity is defined as: “The extent to which an intervention's objectives, design, implementation and impact contribute to, or do not contribute to, addressing systemic inequities and promote a more inclusive society.”

By including “transformative,” the criterion underscores the assumption that reaching equity requires a transformation of systems and structures, both of the evaluand and of the evaluation approach. Systemic inequities refer to the ways systems have been designed, established and maintained that perpetuate inequities.

Transformative equity considers five dimensions:

1. Population/ populace: Who benefits/who loses, Who is included/who is excluded;
2. Cause and effect: How does inequity play out and How is the intervention responding to inequality;
3. Spatial: Where do key inequities persist and what are the geographical and spatial factors affecting equity;
4. Content and intention: What is the transformative change potential of the intervention ? To what extent are interventions designed to contribute to the progressive change for a more equitable South Africa; and
5. Temporal: How has the equity issue changed over time?

Contents

PART A: BACKGROUND, PURPOSE AND FRAMEWORK OF THE GUIDELINE

1 Introduction
2 Purpose of the guideline
3 Understanding inequity in South Africa
4 The importance of mainstreaming transformative equity in evaluation
4.1 The meaning of transformative equity
4.2 Evaluation’s role in promoting transformative equity
5 Transformative equity criterion
6 Equity principles
6.1 Principles supporting the application of equity in evaluations
6.2 Applying equity principles as the commissioner
6.3 Applying equity principles as an evaluator

PART B: INTEGRATING EQUITY INTO EVALUATION: Commissioning, Designing and Undertaking Evaluations

7 Incorporating equity into the evaluation process in a way that encourages the likelihood of use of the evaluation
8 Developing TORs and commissioning of the evaluation
8.1 Determining how far to consider equity in the evaluation
8.2 Considering equity in the focus of the evaluation – purpose, scope and questions
8.2.1 Determining the purpose of an evaluation project
8.2.2 Considering equity in the scope of the evaluation
8.2.3 Determining the evaluation questions
8.3 Evaluation design
8.4 Stakeholder identification and engagement processes
8.5 Budget considerations
8.6 Service provider competencies
9 Evaluation management and implementation
9.1 Managing evaluations
9.2 Evaluation plan
9.3 Fieldwork
9.4 Data analysis
9.5 Validation of findings
10 Bringing equity into the follow-up to the evaluation
10.1 Improvement plan and progress report
10.2 Communicating the results of the evaluation
11. Summary

Annexes

Annex 1: References.
Annex 2: Contributors and reviewers
Annex 3: The purpose of the six types of evaluation
Annex 4: Applying the equity criteria to TORs – case studies

How have you used or intend on using this resource?

I am interested in learning more about the transformative equity approach to equity and how it might be incorporated in all evaluations. I particularly found the cases helpful for suggesting ways to go beyond asking Key Evaluation Questions about average effects to focus on differential effects across different groups, with particular attention to the most vulnerable individuals and communities.

Why would you recommend it to other people?

The guidance goes well beyond simply counting different groups and reporting this and discusses how equity can and should be addressed at all stage of the evaluation process including the commissioning and designing of the evaluation as well as the implementation and reporting.