Determine what ‘success’ looks like
Evaluation is essentially about values, asking questions such as : What is good, better, best? Have things improved or got worse? How can they be improved?
Therefore, it is important for evaluations to be systematic and transparent in the values that are used to decide criteria and standards.
Criteria refer to the aspects of an intervention that are important to consider when deciding whether or not, and in what ways, it has been a success or a failure, or when producing an overall judgement of performance. There are different types of criteria:
Positive outcomes and impacts: for example, should childcare be judged in terms of its success in supporting early childhood development or in supporting parents to engage in education or work? If it is both, how should they be weighted?
Negative outcomes and impacts: for example, an infrastructure development might produce negative unintended effects (e.g. soil erosion caused by a new road) as well as positive intended effects)
Distribution of costs and benefits: for example, is it important for everyone to receive some benefit or the same benefit or for the intervention to be targeted so that the most disadvantaged receive more benefit?
Resources and timing: for example, is there a need for results to be achieved within a certain timeframe?
Processes: for example, use of recyclable materials; providing access to groups with restricted mobility
Standards refer to the levels of performance required for each of the criteria. For example, if a project aims to reduce maternal mortality, what level of performance is needed for it to be considered successful? Any reduction? A reduction of at least xx%? A reduction of at least xx in absolute terms? A reduction to a rate of x.x that matches other similar regions, or matches official targets?
Criteria and standards need to be agreed on in order to identify the data that need to be gathered for an evaluation.
In addition, these data need to be combined to form an overall judgement of success or failure, or to rank alternatives against each other. For example, if a road project achieves its economic objectives but produces environmental damage, should it be considered a success overall? How much damage, and at whose cost, would be enough to outweigh the positive impacts? These issues are addressed under the task Synthesise data from a single evaluation.
Formal statements of values
Some options are used to identify possible criteria and standards that could be used in an evaluation, drawing on formal and informal sources, and some options are used to negotiate which should be used and how they should be weighed.
Articulate and document tacit values
Negotiate between different values
'Determine what ‘success’ looks like' is referenced in:
- Communication for Development (C4D) :