
One way of identifying the values that will be used in an evaluation is to develop explicit standards, evaluative criteria or benchmarks or to use existing relevant standards, criteria or benchmarks.
[Picture: Christopher Meredith Flickr]
‘Standard’ can refer to an aspect of performance, or to the level of performance, or to a combination of both. The level of performance can be specified tightly or described in terms that will vary according to the context. These standards can be considered minimum levels required, or levels required to be considered ‘best practice’.
For example, the SPHERE standards for Humanitarian Aid refer to an aspect of performance (access and water quantity) and to the level of performance (sufficient quantity, sufficiently close) without being prescriptive:
Water supply standard 1: Access and water quantity: All people have safe and equitable access to a sufficient quantity of water for drinking, cooking and personal and domestic hygiene. Public water points are sufficiently close to households to enable use of the minimum water requirement.
By comparison, the Better Business Bureau’s Standards for Charity Accountability refer to an aspect of performance (finance) and to the level of performance (which is tightly defined):
FINANCES: This section of the standards seeks to ensure that the charity spends its funds honestly, prudently and in accordance with statements made in fund raising appeals. To meet these standards, the charitable organization shall 'spend at least 65% of its total expenses on program activities'.
Michael Scriven’s Logic of Evaluation uses the term ‘standards in a different way – it begins by identifying evaluative criteria (aspects of performance), and then criteria (levels of performance), and then collecting evidence of performance and synthesizing it.
Benchmarking involves a process of comparing one’s own performance to an appropriate comparison which might be the industry standard or a similar organization.
Example
The Humanitarian Charter and Minimum Standards for Disaster Response (SPHERE) sets out the standards for the global community to respond to communities affected by disaster. The SPERE project was launched in 1997 by a group of humanitarian NGOs and the Red Cross and Red Crescent movement. The Sphere Project provides a set of guidelines that are set out in the Humanitarian Charter and Minimum Standards in Disaster Response (commonly referred to as the Sphere Handbook).
Hygiene promotion standard 1: Affected men, women and children of all ages are aware of key public health risks and are mobilised to adopt measures to prevent the deterioration in hygienic conditions and to use and maintain the facilities provided.
The Evaluation team that conducted the Independent Completion Report for the Fiji Education Sector Program for AusAID used the five OECD-DAC development evaluation criteria to come to an assessment of the merit and worth of the program. The team added the following three criteria to the usual five:
- Gender Equality
- Monitoring and Evaluation
- Analysis and Learning
Advice
Advice for CHOOSING this option (tips and traps)
- Decide first what you are trying to do, and who are the intended beneficiaries of the results.
Advice for USING this option (tips and traps)
- Be clear about whether you are using benchmarks or not – it can be difficult to get benchmarking data that has been validated by external bodies
- Check whether there are generally accepted standards for performance in this area that should be included
- Be clear of the purpose, audience etc and that if benchmarks are used, they are accepted as valid by the recipients.
Resources
Examples
- Guidebook on responses to humanitarian disasters - The Sphere Handbook, 2011 edition. Each standard outlined in the handbook includes key actions, key indicators and guidance notes. The Handbook is also available in Arabic, German and Russian languages
- Evaluating Development Co-Operation Summary of Key Norms and Standards - A set of core princiciples for evaluation of development assistance adopted by the OECD DAC in 1991 and are at the heart of the Evaluation Network's approach to evaluation. The principles focus on the management and insitutional arrangements of the evaluation system within development agencies
- Fiji Education Sector Independent Completion Report - A 44 page independent evaluation of AusAID's assistance to the Fiji Education sector over the period of 2003-2006
Guide
- ISEAL Standard-Setting Code - Sets the rules for legitimate and effective standard-setting processes, thereby increasing the credibility of the resulting standard. It applies to all standards that promote improvement in social and environmental practices
Sources
Via Sphere Project
OECD DAC Network on Development Evaluation
Via ISEAL Alliance
Comments
Hi
Interesting section and thanks for some revwork. Here are a few thoughts (in no particular order).
Design
Content
Guidance
References
Add some references to benchmarking sites or other material
Hope these thoughts are of some help.
Bye for now
Erica
Add new comment
Login Login and comment as BetterEvaluation member or simply fill out the fields below.