52 weeks of BetterEvaluation: Week 14: Addressing ethical issues
How do we ensure our evaluations are conducted ethically? Where do we go for advice and guidance, especially when we don't have a formal process for ethical review?
On an evaluation discussion list recently there was a question about ethical processes for gathering information from children - in this case an NGO collecting data from children aged 10-18 living in orphanages as part of an evaluation of the effectiveness of programming in the homes.
While large evaluations often have formal structures such as Institutional Review Boards (IRBs) or Ethics Committees to oversee ethical issues, small evaluations sometimes have little support to identify and address ethical issues.
A common issue is confidentiality and anonymity. Be very careful about what you promise and what you can and should deliver. In some cases it is impossible to promise anonymity – such as when data are collected in a group interview or focus group, or when the context of the informant (for example their role as the CEO of the organisation or project manager) makes them readily identifiable even if their name is not used. There may be no private spaces where discussions can be held without others overhearing. Some times it is not appropriate for responses to be anonymous – for example if the informants’ expertise or role in implementation is important in assessing the credibility or relevance of their comment. When it is important to promise anonymity, then make sure data are recorded and stored in ways that will keep the identities of informants concealed.
You should consider other possible risks and costs to those involved in an evaluation - including the value of the time they spend assisting with an evaluation, and possible harm from discussing sensitive issues in an environment which is not safe. It is important to have appropriate procedures to gain informed consent to participate in an evaluation.
One of the 32 tasks in the BetterEvaluation rainbow framework is Define quality evaluation standards, which includes identifying ethical standards to follow. There are several useful resources that can provide guidance to identify and address ethical issues.
Some of these focus specifically on evaluating programs for children - addressing ethical issues and how to address them, particularly in relation to vulnerable children. Many of these include the use of participatory approaches, where the children are actually engaged in the decisions about the evaluation, not just a source of data.
This provides an overview of good practice when involving children in research and evaluation, including a rationale for doing this, ethical issues to address and possible tools to use.
Other useful sources come from evaluation societies and associations which cover small evaluations undertaken by service deliverers as well as large evaluations undertaken by external evaluators.
Some useful resources are below. Are there others you would recommend - or examples of good practice we should add?
This paper from the American Evaluation Association (AEA) outlines the guiding principles to be used by evaluators in order to promote ethical practice in evaluations.
This paper from the African Evaluation Association (AfrEA) provides a brief description of the guidelines and a series of checklists to assist with the planning, implementation and completion of the evaluation process.
The guidelines are divided into four key areas:
This statement of the American Evaluation Association (AEA) affirms the significance of cultural competence in evaluation. It also informs the public of AEA’s expectations concerning cultural competence in the conduct of evaluation.