52 weeks of BetterEvaluation: Week 11: Using rubrics
The term "rubric" is often used in education to refer to a systematic way of setting out the expectations for students in terms of what would constitute poor, good and excellent performance.
In recent years rubrics have begun to be used explicitly in evaluation to address the challenge of systematically and transparently synthesising diverse evidence into an overall evaluative judgement. Sometimes they are called rubrics, and sometimes Global Assessment Scales, as they provide an overall rating of performance, based on detailed descriptions on a scale.
The main point of a rubric is that it is designed to support a holistic judgement about performance. It is not a scale that involves awarding points for this and points for that and then adding them up and reading off the scale.
There are now examples of rubrics being used in many sectors, including agriculture (rating community groups involved in a dryland salinity project), employment (rating the success of a program for seasonal immigrant workers from the point of view of both the workers and the employers), and Indigenous education (12 separate rubrics developed for different aspects of the Measurable Gains Framework in the Maori Education Strategy).
Rubrics can be used to ensure comparability of ratings across sites and raters and to effectively synthesise diverse evidence. They can also have important impacts through the process of developing and using them - negotiating agreement about criteria standards, and appropriate evidence of performance, and focusing attention on what is highly valued.
On the BetterEvaluation site, we have links to some great resources for developing rubrics, including guides and examples, which you can access through the options page on rubrics. But many questions remain.
That is why we're delighted that this week, when we launch our series of evaluation cases (developed through a writeshop process led by Irene Guijt) the first case will focus on an evaluation where rubrics were used.
Judy Oakden's case focuses on the use of rubrics in the evaluation of a leadership development program for principals - discussing how the rubric was developed, what it looked like and how it was used. In addition, her case sets the use of rubrics within the context of the whole evaluation - how it was managed, the framing of the evaluation in terms of its purpose and key questions, the theory of change underpinning the program, how data were collected and analysed, and what was done to report findings and support their use.
Watch for her guest blog post, the evaluation case and the chance to ask questions about developing and using rubrics in evaluation.
Image source (top): New Zealand Qualifications Authority (2009) Policy and Guidelines for the Conduct of External Evaluation and Review