Peer-assessment

Examples

UKES in collaboration with EES is conducting a pilot of voluntary evaluator peer review (VEPR) (UKES 2014). The approach does not seek to judge a candidates' evaluation capabilities or competence, but rather their willingness to submit to a professional review structured around a capabilities framework.”

Applicants are those with:

• Evidence of minimum two years equivalent work in evaluation and/or evaluation related activities
• Significant involvement (based on role e.g. purchaser, provider or manager) in a diverse portfolio of evaluation projects
• Evidence of relevant academic qualifications and previous professional development initiatives

The following process is used:

• The applicant nominates two preferred peer reviewers from the VEPR reviewer 'pool'
• The applicant submits a completed application nominating (1) 2-3 areas of evaluation practice on which they wish to focus and (2) two recent projects on which to focus the review (note: this may require commissioners' and/or colleagues' permission)
• A standardised review process will be based on a set of questions that require the applicant to address selected capabilities in the society's capability framework, based on their current and recent practice, within a review meeting of approximately two hours (ideally face-to-face)
• Applicant and reviewers consult to compile a list of agreed questions for the applicant to address
• Applicant submits short written answers to each question of the two reviewers
• The review takes place either face-to-face or via Skype or teleconference
• Where satisfactory, reviewed applicants' names listed on an online VEPR Index (see below) on the society's website, as evidence of a 'satisfactory' review
• Listing on the VEPR Index lapses automatically after three years unless renewed via a further VEPR.

Advice for using this method

• Guidelines on how to give and receive feedback during peer review should be discussed or formalised before beginning a review process. This can include a set of agreed questions for the review, or identification of competencies to be addressed.
• Organisations, evaluation associations and societies may consider creating a peer-review assessment tool and supporting members to take this up on a voluntary basis.