C4D Hub: Define ethical and quality evaluation standards for R,MandE

What is it?

It is important to be agree on and be clear about what both the quality standards are for the R,M&E (issues such as rigor, contextuality, gender sensitivity, impartiality and other criteria about the quality of R,M&E), and what the ethical standards are (being respectful, sensitive, transparent and avoiding causing harm or raising false expectations). Quality and ethical standards should be agreed to early on, and adhered to throughout implementation. The ethical standards become particularly relevant considering methods and the workplan. It is also important to ensure there are processes to maintain awareness among all stakeholders about the agreed standards - for example by including a planned review process of the evaluation at key stages (e.g. the design, and the draft report - see Review R,MandE systems and studies (meta evaluation).

General information on Quality Standards

The main BetterEvaluation site covers general guidance on agreeing to quality and ethical standards, with links to information on ethical guidelines and qualities standards. The UN Evaluation Group (UNEG) (to which UNICEF adheres), offers several guidance documents, including the UNEG Norms and Standards, the UNEG Quality Checklist for Evaluation Reports, and the UN Code of Conduct for Evaluation in the UN System. (click here for all other UNEG guidance documents). There are 13 ‘Norms’ in the UNEG Norms and Standards, which range from stating that UN agencies should have evaluation policies in place, to discussions around impartiality, independence and ethics. There are four ‘Standards’ (which overlap somewhat with the 13 Norms), each with a series of sub-level standards. These pages and resources are recommended background reading before considering options to apply to C4D.  

General information on Ethics in R,M&E

Ethics can feel quite challenging, but it really comes down to being respectful, transparent and avoiding causing harm. Important to identify and ethical risks and develop strategies and processes for managing these. There are many resources available on this topic. The site is a comprehensive source for recommended resources. Below are some with a particular focus on children and adolescents.

Ethical and quality standards for R,M&E and C4D

Applying the C4D principles

Accountable 

The quality and ethical standards for C4D R,M&E should reflect the expectations of all the people and groups we are accountable to (donors and managers, partners and community groups). Defining and following quality and ethical standards is important for maintaining accountability and integrity in RM&E. Ensuring ethical practices in RM&E is a responsibility of everyone involved in the R,M&E.  

Critical 

It is important to question existing sets of standards and their relevance in the local setting. We need to ask: whose interests and expectations are reflected in the quality and ethical standards? what are the assumptions embedded in the standards? what other perspectives are missing from those standards? 

 

Participatory 

Participatory processes can be used to develop and clarify quality and ethical standards with partners, and community groups. This ensures that standards lay out appropriate practices in keeping with local standards and expectations. In terms of ethical standards, participatory approaches to define ethical standards can help ensure these are locally appropriate, especially where participatory methods are used, or where sensitive topics are being explored.

Holistic 

Our expectations and perceptions of quality and ethics are culturally bound. In seeking agreement on quality and ethical standards it is important to understand these in the context of social, cultural, and organisational systems. 

Realistic

In C4D the ethical standards should cover sharing results and findings in accessible ways (especially with marginalised groups and those who were consulted in the data collection and report writing process) as an ethical responsibility. This also helps with promoting a learning-based culture and continuous learning.

Recommended options and adaptations for Quality Standards and C4D

Independence and impartiality and perceived conflicts

Where there is a perceived conflict between participatory approaches the R,M&E and the Quality Standards (i.e. independence and impartiality), it may be useful to think of the range of options for including stakeholders in participatory ways, including in decision making. The Participation Matrix can be a useful way to balance the quality standards and participatory approaches.  

Example

My Rights My Voice Completion Report offers a summary of the Global Evaluation, which consisted of an extensive analysis of programme documents for all 8 My Right My Voice (MRMV) countries.  Click here to go directly the My Rights My Voice report (see page 52-53), or read a summary page about this exemplar. This example is consistent with the C4D Evaluation framework in relation to this task in the following ways:

  • Participatory: the evaluation was by a team of independent evaluators, but worked closely to support youth familiar with the programme to independently carry out evaluation research with peers, parents and teachers, and present the findings to Oxfam staff and partners. It is an example of participatory research that incorporates strategies to ensure quality standards (such as independence and impartiality) are maintained. 

General Resources on Ethics in R,M&E

So You Want to Involve Children in Research? A Toolkit Supporting Children’s Meaningful and Ethical Participation in Research Relating to Violence Against Children. This practical toolkit by Laws and Mann for Save the Children (2004).This toolkit includes key principles, case studies and a checklist of key ethical considerations in M&E involving children.  http://resourcecentre.savethechildren.se/library/so-you-want-involve-children-research-toolkit-supporting-childrens-meaningful-and-ethical

Participatory Approaches by Irene Guijt (2014) is part of a series of Methodological Briefs by the UNICEF Office of Research. It offers comprehensive guidance on involving children in participatory M&E, including a checklist of key ethical concerns. For the full guidelines, see http://ppfcuk.files.wordpress.com/2012/02/ppfccoreprinciples.pdf)

Ethical Research Involving Children - a compendium put together by UNICEF with a range of other partners, covering harms and benefits, informed consent, privacy and confidentiality, payment and compensation, a section on available supports, sections on the different stages (planning, design, data collection, analysis & dissemination etc.), and finishing with a long list of case studies. https://www.unicef-irc.org/publications/706/ 

Evaluation Technical Notes - from UNICEF Evaluation Office 2002. Provides an overview of ethical considerations when involving children in M&E. https://www.unicef.org/evaluation/files/TechNote1_Ethics.pdf 

Doing Qualitative Field Research on Gender Norms with Adolescent Girls and Their Families includes practical advice, examples and tools to ensure gender sensitivity in evaluation and research with adolescent girls. Click here to go directly to the resource, or read the full summary and review of this resource. It is consistent with the C4D Evaluation Framework in the following ways:

  • critical: the guide takes seriously the gender specific considerations that are required for ethical evaluation research and provides practical tools
  • participation: using this guide will help ensure meaningful participation and voice by adolescent girls in evaluation research.
  • holistic: the guide suggests open-ended questions and including family members in research to bring a holistic understanding.  

Oxfam Responsible Program Data PolicyThis document outlines a rights-based policy for ethical data management (see also Manage Data), based on the following rights: the right to be counted and heard; the right to dignity and respect; the right to make an informed decision; the right to privacy; and the right to not be put at risk. This policy is consistent with the C4D Evaluation Framework in the following ways:

  • CriticalThe policy recognises that data and ownership of data entails a position of power and responsibility, and the importance of the rights of marginalised groups in this process.  
  • Accountable: The policy emphasises the ethical dimensions of data management processes and responsibilities.

Specific resources on ethics and C4D R,M&E

Resources

Module 6 of the Equal Access Participatory M&E toolkit ‘Getting Started and Planning for PM&E and Impact Assessment’ (see the full toolkit)

Pages 15-18 includes discussion about dealing with gender, literacy and sensitive issues, as well as ethical issues in C4D research monitoring and evaluation. 

The IDEAS Guide (Module 5 and 8) includes activities to learn about common ethical risks in media and communication projects and evaluation, and to reflect on how these relate to projects. This resource is particularly good as an entry level guide.

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.