Standards, evaluative criteria and benchmarks

Synonyms: 
Benchmarks and standards

One way of identifying the values that will be used in an evaluation is to develop explicit standards, evaluative criteria or benchmarks or to use existing relevant standards, criteria or benchmarks.

[Picture: Christopher Meredith Flickr]

‘Standard’ can refer to an aspect of performance, or to the level of performance, or to a combination of both. The level of performance can be specified tightly or described in terms that will vary according to the context. These standards can be considered minimum levels required, or levels required to be considered ‘best practice’.

For example, the SPHERE standards for Humanitarian Aid refer to an aspect of performance (access and water quantity) and to the level of performance (sufficient quantity, sufficiently close) without being prescriptive:

Water supply standard 1: Access and water quantity: All people have safe and equitable access to a sufficient quantity of water for drinking, cooking and personal and domestic hygiene. Public water points are sufficiently close to households to enable use of the minimum water requirement.

By comparison, the Better Business Bureau’s Standards for Charity Accountability refer to an aspect of performance (finance) and to the level of performance (which is tightly defined):

FINANCES: This section of the standards seeks to ensure that the charity spends its funds honestly, prudently and in accordance with statements made in fund raising appeals. To meet these standards, the charitable organization shall 'spend at least 65% of its total expenses on program activities'.

Michael Scriven’s Logic of Evaluation uses the term ‘standards in a different way – it begins by identifying evaluative criteria (aspects of performance), and then criteria (levels of performance), and then collecting evidence of performance and synthesizing it.

Benchmarking involves a process of comparing one’s own performance to an appropriate comparison which might be the industry standard or a similar organization.

Example

The Humanitarian Charter and Minimum Standards for Disaster Response (SPHERE) sets out the standards for the global community to respond to communities affected by disaster. The SPERE project was launched in 1997 by a group of humanitarian NGOs and the Red Cross and Red Crescent movement. The Sphere Project provides a set of guidelines that are set out in the Humanitarian Charter and Minimum Standards in Disaster Response (commonly referred to as the Sphere Handbook). 

Hygiene promotion standard 1: Affected men, women and children of all ages are aware of key public health risks and are mobilised to adopt measures to prevent the deterioration in hygienic conditions and to use and maintain the facilities provided.

The Evaluation team that conducted the Independent Completion Report for the Fiji Education Sector Program for AusAID used the five OECD-DAC development evaluation criteria to come to an assessment of the merit and worth of the program. The team added the following three criteria to the usual five: 

  • Gender Equality
  • Monitoring and Evaluation
  • Analysis and Learning

Advice

Advice for CHOOSING this option (tips and traps)

  • Decide first what you are trying to do,  and who are the intended beneficiaries of the results.

Advice for USING this option (tips and traps)

  • Be clear about whether you are using benchmarks or not – it can be difficult to get benchmarking data that has been validated by external bodies
  • Check whether there are generally accepted standards for performance in this area that should be included
  • Be clear of the purpose, audience etc and that if benchmarks are used, they are accepted as valid by the recipients.

Resources

Examples

Guide

  • ISEAL Standard-Setting Code - Sets the rules for legitimate and effective standard-setting processes, thereby increasing the credibility of the resulting standard. It applies to all standards that promote improvement in social and environmental practices

Sources

Via Sphere Project   

OECD DAC Network on Development Evaluation

Via ISEAL Alliance

Via Better Business Bureaus

Updated: 6th March 2014 - 12:01pm
This Option is useful for:
A special thanks to this page's contributors
Author
Director of BetterEvaluation/ Professor of Public Sector Evaluation, Australia and New Zealand School of Government.
Melbourne.

Comments

ericadyson's picture
Erica Dyson

Hi

Interesting section and thanks for some revwork. Here are a few thoughts (in no particular order).

Design

  • the font color and size make it difficult to read.. and wide margins.. so personally I would use more color, larger font and narrower margins for when reading .. ie make it less dense
  • overall layout could do with a review ... ie on the basis that it is too dense especially since the subject matter itself is quite 'dry'

Content

  • didn't quite grasp how the OECD evaluation criteria section relates to standards.. perhaps there needs to be a 'hit you over the head' couple of sentences that make the connection clear
  • why is adding criteria,a matter of standards, or benchmarks
  • perhaps add some example of benchmarks on the same topic as water.. ie to compare and contrast

Guidance

  • add be clear how you are going to measure your work against the standard.. and check with the organization issuing the standards if they have best practice on how to do this

References

Add some references to benchmarking sites or other material

Hope these thoughts are of some help.

Bye for now

Erica

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.