Request for feedback: What should go into an evaluation report rating scale?

georgeargyrous's picture 6th December 2018 by georgeargyrous

As part of a project with an Australian state government agency, I am developing a rubric for people with little to no evaluation skills who might need to judge the quality of an evaluation report. This is within the context of a larger project whereby an evidence base of past evaluation reports is made available for program designers. We want users to access these reports, but also have some support to judge the quality of the reports.

I have searched a number of existing tools and assembled an (over-inclusive) list of items that could be part of an evaluation report quality rubric. Items are grouped around common themes that existing rubrics use. There are also some comments that help explain each item. It is anticipated that each item in the rubric will be rated on a simple No/Partially/Yes/Not applicable scale.

Feedback wanted:

The questions I have that I would like feedback upon, are:

  1. Are these the appropriate categories for organising the items?
  2. Are the items appropriately worded?
  3. Are some unnecessary?
  4. Are some missing?
  5. What other comments are needed to help explain the items?

Click to view rubric

About this rubric:

Who is the audience?

The rubric is being designed for users that will include people who have very limited evaluation experience. This will include people designing programs within the relevant policy space.

What is the aim of the rubric?

We want users to be able to be critical consumers of evaluation reports. This rubric aims to provide users with a minimum set of criteria to judge the quality of evaluation reports.

How will this rubric be used?

This rubric will be made available alongside an online evidence database containing a range of evaluation reports. It is hoped that users of the database will be able to use the checklist to assess the quality of the reports in the database, rather than simply drawing upon these reports without thinking critically about the quality.

Share your feedback

Please leave a comment below to give feedback on the rubric. You can also use the BetterEvaluation contact form to give feedback privately, and this will be passed on.

Thank you!

 

A special thanks to this page's contributors
Author
Senior Lecturer, Australia and New Zealand School of Government.
Australia.

Comments

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.