RQ+ Research Quality Plus. A Holistic Approach to Evaluating Research

This report describes a holistic approach and assessment framework for evaluating 'research' that goes beyond the traditional deliberative means (e.g., peer review) and often used analytics (e.g., bibliometrics).

The approach underscores that scientific merit is a necessary but insufficient condition for judging research quality and acknowledges that determining the effectiveness of research (in terms of its relevance, use and impact) should not only involve researchers, research managers and funders but multiple stakeholders and potential users. Hence, the RQ+ Assessment Framework consists of three components:

(1) key influences (enabling or constraining factors) either within the research endeavor or in the external environment;

(2) research quality dimensions (integrity, legitimacy, importance, positioning for use) and subdimensions which are closely inter-related; and,

(3) customizable assessment rubrics (also called 'evaluative rubrics') that make use of both qualitative and quantitative measures.

The work was conducted with input from staff and support from the International Development Research Centre (IDRC) in Canada which primarily funds and facilitates global South-based research for development.

The information provided was supplied by Robert McLean, IDRC, and Andréanne Martel, IDRC.

Authors and their affiliation

Zenda Ofir - Independent Evaluator & Honorary Professor, University of Stellenbosch, South Africa

Thomas Schwandt - Professor, University Of Illinois, Urbana-Champaign, USA

Colleen Duggan - Senior Program Specialist, Policy and Evaluation Division, International Development Research Centre, Canada

Robert McLean - Senior Program Specialist, Policy and Evaluation Division, International Development Research Centre, Canada & Lead Evaluator, Canadian Institutes of Health Research, Canada

Year of publication 

2016

Type of resource

Tool

Key features

Interest and debate about the quality and use of research, as well as the relationship of research undertakings to social goals and values, have been particularly intense in the field of internationally funded research on the development priorities of low-to-middle-income countries. It is in this arena -research for development (R4D)- that the approach to evaluating research quality discussed in this report was designed.

This report describes, in brief, the rationale behind the development of this new approach to evaluating research quality. It then discusses in some detail the assessment instrument itself, followed by a section on lessons learned from the first effort to implement it in the External Reviews of the IDRC work. In conclusion, the report reflects on potential uses of the instrument as well as ideas for its further refinement.

Who is this resource useful for?

  • Commissioners/managers of evaluation;
  • Evaluators;
  • Those involved in evaluation capacity strengthening;
  • Funders of research;
  • Researchers engaged in evaluating research quality.

How have you used or intend on using this resource?

What is discussed here is the inaugural version of the RQ+ Assessment Framework. It is currently undergoing revision and adaptation by IDRC for its own specific circumstances and uses. Readers are encouraged to treat this Framework as a dynamic, evolving tool that they can adapt and modify for their specific purposes.

The authors expect that this approach to evaluating research quality will be improved by the practitioners who use the Framework and through further examination by scholars concerned with issues of research quality and use.

Why would you recommend it to other people?

This RQ+ approach holds potential as an evaluation method, but also as a method for enhancing research management. As an evaluation tool it can help one to systematically evaluate research quality. As a research management tool, RQ+ can support individual research project, but might also serve as a tool for multi-project research programming.

RQ+ may be most useful for “policy and practice-oriented research”.  Particularly, research that encourages a wide variety of actors in the knowledge production process, conducted by researchers concerned with the potential social benefit of their research.

Furthermore, unlike other tools to evaluate research quality, RQ+ is adapted to the complexity and specific nature of research in the field of international development.

Contents

Introduction

Part 1: Rationale

Part 2: The RQ+ Assessment Framework - this is the inaugural version of the framework. IDRC is currently making adaptations for its own circumstances and uses but also encourages others to treat the framework as a dynamic, evolving tool to modify to their specific purposes.

Part 3: Lessons from the implementation of the RQ+ Assessment Framework by IDRC

Part 4: The Potential of the RQ+ Assessment Framework

References

  

Source

Ofir, Z., T. Schwandt, D. Colleen, and R. McLean (2016). RQ+ Research Quality Plus. A Holistic Approach to Evaluating Research. Ottawa: International Development Research Centre (IDRC).

0
No votes yet
Rate this Resource:
This resource is useful for:
A special thanks to this page's contributors
Resource Suggested By
Senior Program Specialist, IDRC.
Ottawa, Canada.
Resource Suggested By
Research Awardee, International development research centre (IDRC).
Ottawa, Canada.

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.