Standards, evaluative criteria and benchmarks

‘Standard’ can refer to an aspect of performance, or to the level of performance, or to a combination of both. The level of performance can be specified tightly or described in terms that will vary according to the context. These standards can be considered minimum levels required, or levels required to be considered ‘best practice’.

For example, the SPHERE standards for Humanitarian Aid refer to an aspect of performance (access and water quantity) and to the level of performance (sufficient quantity, sufficiently close) without being prescriptive:

Water supply standard 1: Access and water quantity: All people have safe and equitable access to a sufficient quantity of water for drinking, cooking and personal and domestic hygiene. Public water points are sufficiently close to households to enable use of the minimum water requirement.

By comparison, the Better Business Bureau’s Standards for Charity Accountability refer to an aspect of performance (finance) and to the level of performance (which is tightly defined):

FINANCES: This section of the standards seeks to ensure that the charity spends its funds honestly, prudently and in accordance with statements made in fund raising appeals. To meet these standards, the charitable organization shall 'spend at least 65% of its total expenses on program activities'.

Michael Scriven’s Logic of Evaluation uses the term ‘standards in a different way – it begins by identifying evaluative criteria (aspects of performance), and then criteria (levels of performance), and then collecting evidence of performance and synthesizing it.

Benchmarking involves a process of comparing one’s own performance to an appropriate comparison which might be the industry standard or a similar organization.

Example

The Humanitarian Charter and Minimum Standards for Disaster Response (SPHERE) sets out the standards for the global community to respond to communities affected by disaster. The SPERE project was launched in 1997 by a group of humanitarian NGOs and the Red Cross and Red Crescent movement. The Sphere Project provides a set of guidelines that are set out in the Humanitarian Charter and Minimum Standards in Disaster Response (commonly referred to as the Sphere Handbook). 

Hygiene promotion standard 1: Affected men, women and children of all ages are aware of key public health risks and are mobilised to adopt measures to prevent the deterioration in hygienic conditions and to use and maintain the facilities provided.

The Evaluation team that conducted the Independent Completion Report for the Fiji Education Sector Program for AusAID used the five OECD-DAC development evaluation criteria to come to an assessment of the merit and worth of the program. The team added the following three criteria to the usual five: 

• Gender Equality
• Monitoring and Evaluation
• Analysis and Learning

Advice

Advice for CHOOSING this option (tips and traps)

• Decide first what you are trying to do,  and who are the intended beneficiaries of the results.

Advice for USING this option (tips and traps)

• Be clear about whether you are using benchmarks or not – it can be difficult to get benchmarking data that has been validated by external bodies
• Check whether there are generally accepted standards for performance in this area that should be included
• Be clear of the purpose, audience etc and that if benchmarks are used, they are accepted as valid by the recipients.

Resources

Examples

• Guidebook on responses to humanitarian disasters - The Sphere Handbook, 2011 edition. Each standard outlined in the handbook includes key actions, key indicators and guidance notes. The Handbook is also available in Arabic, German and Russian languages
• Evaluating Development Co-Operation Summary of Key Norms and Standards - A set of core princiciples for evaluation of development assistance adopted by the OECD DAC in 1991 and are at the heart of the Evaluation Network's approach to evaluation. The principles focus on the management and insitutional arrangements of the evaluation system within development agencies
• Fiji Education Sector Independent Completion Report - A 44 page independent evaluation of AusAID's assistance to the Fiji Education sector over the period of 2003-2006

Guide

• ISEAL Standard-Setting Code - Sets the rules for legitimate and effective standard-setting processes, thereby increasing the credibility of the resulting standard. It applies to all standards that promote improvement in social and environmental practices

Sources

Via Sphere Project   

OECD DAC Network on Development Evaluation

Via ISEAL Alliance

Via Better Business Bureaus