RQ+ Research Quality Plus. A Holistic Approach to Evaluating Research

This report describes a holistic approach and assessment framework for evaluating 'research' that goes beyond the traditional deliberative means (e.g., peer review) and often used analytics (e.g., bibliometrics).

The information provided was supplied by Robert McLean, IDRC, and Andréanne Martel, IDRC.

Authors and their affiliation

Zenda Ofir - Independent Evaluator & Honorary Professor, University of Stellenbosch, South Africa

Thomas Schwandt - Professor, University Of Illinois, Urbana-Champaign, USA

Colleen Duggan - Senior Program Specialist, Policy and Evaluation Division, International Development Research Centre, Canada

Robert McLean - Senior Program Specialist, Policy and Evaluation Division, International Development Research Centre, Canada & Lead Evaluator, Canadian Institutes of Health Research, Canada

Year of publication 

2016

Type of resource

Tool

Key features

Interest and debate about the quality and use of research, as well as the relationship of research undertakings to social goals and values, have been particularly intense in the field of internationally funded research on the development priorities of low-to-middle-income countries. It is in this arena -research for development (R4D)- that the approach to evaluating research quality discussed in this report was designed.

This report describes, in brief, the rationale behind the development of this new approach to evaluating research quality. It then discusses in some detail the assessment instrument itself, followed by a section on lessons learned from the first effort to implement it in the External Reviews of the IDRC work. In conclusion, the report reflects on potential uses of the instrument as well as ideas for its further refinement.

Who is this resource useful for?

• Commissioners/managers of evaluation;
• Evaluators;
• Those involved in evaluation capacity strengthening;
• Funders of research;
• Researchers engaged in evaluating research quality.

How have you used or intend on using this resource?

What is discussed here is the inaugural version of the RQ+ Assessment Framework. It is currently undergoing revision and adaptation by IDRC for its own specific circumstances and uses. Readers are encouraged to treat this Framework as a dynamic, evolving tool that they can adapt and modify for their specific purposes.

The authors expect that this approach to evaluating research quality will be improved by the practitioners who use the Framework and through further examination by scholars concerned with issues of research quality and use.

Why would you recommend it to other people?

This RQ+ approach holds potential as an evaluation method, but also as a method for enhancing research management. As an evaluation tool it can help one to systematically evaluate research quality. As a research management tool, RQ+ can support individual research project, but might also serve as a tool for multi-project research programming.

RQ+ may be most useful for “policy and practice-oriented research”.  Particularly, research that encourages a wide variety of actors in the knowledge production process, conducted by researchers concerned with the potential social benefit of their research.

Furthermore, unlike other tools to evaluate research quality, RQ+ is adapted to the complexity and specific nature of research in the field of international development.