Evaluating humanitarian action

Theme type:
Synonyms:
EHA

Different types of evaluation are used in humanitarian action for different purposes, including rapid internal reviews to improve implementation in real time and discrete external evaluations intended to draw out lessons learned with the broader aim of improving policy and practice, and enhancing accountability.

Humanitarian action is any activity taken with the objective of saving lives, alleviating suffering, and maintaining human dignity during and after human-induced crises and disasters resulting from natural hazards. Humanitarian action also includes prevention and preparation for these.  Humanitarian action includes both the provision of assistance (such as food, healthcare and shelter) and the protection of crisis-affected populations from violations of their rights (as defined by human rights law, international humanitarian law, and refugee law, see ALNAP, 2016).

Different types of evaluation are used in humanitarian action for different purposes, including rapid internal reviews to improve implementation in real time and discrete external evaluations intended to draw out lessons learned with the broader aim of improving policy and practice, and enhancing accountability. The evaluation of humanitarian action (EHA) mostly focuses on evaluating humanitarian projects or programmes funded by an individual donor, although some work has also evaluated the multiple efforts of several actors in response to the same crisis.

What is different about EHA?

Evaluation practitioners in humanitarian contexts have a need for specific learning opportunities and support as evaluative challenges are often accentuated in these difficult environments. A few of the most common challenges include:

  1. Constrained access: speaking to affected populations may be challenging, limited or impossible. The evaluator may not be able to visit projects or programmes, thus doing most of the evaluation remotely.
  2. Lack of data: data may have been destroyed or made irrelevant because of conflict or population movements. Baseline data may be hard to come by, particularly in more protracted crises.
  3. Rapid and chaotic responses: Projects or programmes may not have clear project plans or Theories of Change. Work was planned quickly and needed to evolve to suit changes in the crisis. 
  4. High staff turnover: humanitarian projects have tended to be shorter compared to projects in other sectors, such as international development. Staff may not stay very long within a response. Finding key informants can be challenging for evaluators. 
  5. Data protection and ethical considerations: It is difficult to design data collection and management tools that meet the ethical and analytical challenges raised by ‘do no harm’ principles and protection risk reduction.

Subsequently, practitioners of EHA continue to struggle with producing strong evidential quality of humanitarian evaluations (ALNAP, 2018). To help address this, several coordination networks and learning resources now exist for humanitarian evaluation practitioners, including ALNAP’s Community of Practice and Evaluation of Humanitarian Action Guide

How to do EHA

The importance of EHA and discussion of its particular features and challenges is increasingly recognised. Indeed, the Development Assistance Committee (DAC) of the Organisation for Economic Co-operation and Development (OECD) refined its original four principles for the evaluation process for DAC members into seven criteria that have been adapted for evaluation of complex emergencies (please see the list below). These are typically used as the ‘industry standard’ for EHA.

  1. Relevance/appropriateness
  2. Connectedness
  3. Coherence
  4. Coverage
  5. Efficiency
  6. Effectiveness
  7. Impact

EHA can use a range of evaluative tools, from After-Action Reviews to discrete impact evaluations. Examples of these are presented in the spectrum below, extracted from the ALNAP EHA Guide (note that on the BetterEvaluation website, all of what ALNAP refers to as ‘evaluative tools’ in this diagram would fall under the umbrella term ‘evaluation’, which covers the full range of approaches to monitoring and evaluation).

Examples

Resources

Guides

Online course

Websites

Discussion papers

Blogs

ALNAP (2016) Evaluation of Humanitarian Action Guide. ALNAP Guide. London: ALNAP/OI.

ALNAP (2018) The state of the humanitarian system 2018. London: ALNAP/ODI

Feature image: An aid worker collects health and (mal)nutrition data during a field visit in Mandera, northeastern Kenya. July 2009. Malnutrition is a big problem among children under 5 in this arid border town. Source: marlenefrancia / Shutterstock.com 

'Evaluating humanitarian action' is referenced in: