52 weeks of BetterEvaluation: Week 14: Addressing ethical issues

Patricia Rogers's picture 5th April 2013 by Patricia Rogers

How do we ensure our evaluations are conducted ethically? Where do we go for advice and guidance, especially when we don't have a formal process for ethical review?

On an evaluation discussion list recently there was a question about ethical processes for gathering information from children - in this case an NGO collecting data from children aged 10-18 living in orphanages as part of an evaluation of the effectiveness of programming in the homes.

While large evaluations often have formal structures such as Institutional Review Boards (IRBs) or Ethics Committees to oversee ethical issues, small evaluations sometimes have little support to identify and address ethical issues.

A common issue is confidentiality and anonymity. Be very careful about what you promise and what you can and should deliver. In some cases it is impossible to promise anonymity – such as when data are collected in a group interview or focus group, or when the context of the informant (for example their role as the CEO of the organisation or project manager) makes them readily identifiable even if their name is not used. There may be no private spaces where discussions can be held without others overhearing. Some times it is not appropriate for responses to be anonymous – for example if the informants’ expertise or role in implementation is important in assessing the credibility or relevance of their comment. When it is important to promise anonymity, then make sure data are recorded and stored in ways that will keep the identities of informants concealed.

You should consider other possible risks and costs to those involved in an evaluation - including the value of the time they spend assisting with an evaluation, and possible harm from discussing sensitive issues in an environment which is not safe. It is important to have appropriate procedures to gain informed consent to participate in an evaluation.

Useful resources

One of the 32 tasks in the BetterEvaluation rainbow framework is Define quality evaluation standards, which includes identifying ethical standards to follow. There are several useful resources that can provide guidance to identify and address ethical issues.

Some of these focus specifically on evaluating programs for children - addressing ethical issues and how to address them, particularly in relation to vulnerable children.  Many of these include the use of participatory approaches, where the children are actually engaged in the decisions about the evaluation, not just a source of data.

Useful Tools for Engaging Young People in Participatory Evaluation

A toolkit developed as part of a formative evaluation by the UNICEF Regional Office of young people's participation in the CEE/CIS region in 2005. It includes an overview of ethical issues and guidance on using 15 different evaluation tools/options with children, plus icebreakers and energizers to use in group activities.

Children and Participation: Research, monitoring and evaluation with children and young people

This provides an overview of good practice when involving children in research and evaluation, including a rationale for doing this, ethical issues to address and possible tools to use.

Ethical Approaches to Gathering Information from Children and Adolescents in International Settings

This provides guidance in terms of basic ethical principles:

Respect for persons: A competent individual has the right to self-determination. Beneficence: An investigator must protect participants from harm and maximize possible benefits.

Justice: The benefits and burdens of the activity should be equitably distributed.

Specific guidelines for information gathering activities are provided with reference to the following questions:

These guidelines are presented as questions and responses within six subsections:

1. Is this activity necessary and justified?

2. Is this activity designed to get valid information?

3. Have community and stakeholder concerns been considered?

4. Have adverse consequences of the activity been anticipated?

5. Are procedures in place to ensure that the activity proceeds ethically?

6. Is there a clear plan and adequate funding for follow-up activities?

Other useful sources come from evaluation societies and associations which cover small evaluations undertaken by service deliverers as well as large evaluations undertaken by external evaluators. 

Some useful resources are below.  Are there others you would recommend - or examples of good practice we should add?

AEA ethical guidelines

This paper from the American Evaluation Association (AEA) outlines the guiding principles to be used by evaluators in order to promote ethical practice in evaluations.

  • Systematic Inquiry: Evaluators conduct systematic, data-based inquiries
  • Competence: Evaluators provide competent performance to stakeholders
  • Integrity/Honesty: Evaluators display honesty and integrity in their own behaviour, and attempt to ensure the honesty and integrity of the entire evaluation process
  • Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders
  • Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values

AES ethical guidelines

Ethics refers to right and wrong in conduct. These are guidelines for ethical behaviour and decision making in evaluation. They are intended firstly to promote the ethical practice of evaluation. Through stimulating awareness and discussion of ethical issues, the guidelines aim to foster continuing improvement in the theory, practice and use of evaluation.

AfrEA ethical guidelines

This paper from the African Evaluation Association (AfrEA) provides a brief description of the guidelines and a series of checklists to assist with the planning, implementation and completion of the evaluation process.

The guidelines are divided into four key areas:

  1. Utility: The utility guidelines are intended to ensure that an evaluation will serve the information needs of intended users and be owned by stakeholders.
  2. Feasibility: The feasibility guidelines are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal.
  3. Propriety - The propriety guidelines are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.
  4. Accuracy - The accuracy guidelines are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth of merit of the program being evaluated. 

Statement on Cultural Competence in Evaluation Tool

This statement of the American Evaluation Association (AEA) affirms the significance of cultural competence in evaluation. It also informs the public of AEA’s expectations concerning cultural competence in the conduct of evaluation.

Image: Chris interview 2 by transaid images, on Flickr

52 Weeks of BetterEvaluation. Click here to view past features

A special thanks to this page's contributors
Author
CEO, BetterEvaluation.
Melbourne.

Comments

Lucy J Callaghan's picture
Lucy J Callaghan

These resources are really helpful and provided terrific guidance in preparation for gathering information from children and young people as part of an independent evaluation examining a service designed and implemented for them. Thanks!

Emma Smith's picture
Emma Smith

Fantastic to hear, Lucy. 

Thanks for letting us know. 

All the best, 

Emma

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.