What methods may be used in impact evaluations of humanitarian assistance?

This 3ie working paper examines the extent to which impact evaluation methods can provide evidence to help improve the effectiveness and efficiency in humanitarian action.

This resource and the following information was contributed by Amelie Sundberg, Neil Dillon, and Maria Gill.

Authors and their affiliation

• Jyotsna Puri, International Initiative for Impact Evaluation (3ie) 
• Anastasia Aladysheva, Stockholm International Peace Research Institute (SIPRI) 
• Vegard Iversen, University of Manchester 
• Yashodhan Ghorpade, Institute of Development Studies 
• Tilman Brück, SIPRI and Institute for the Study of Labour (IZA)

Key features

This paper aims to examine the extent to which scientific impact evaluation methods can provide evidence to improve the effectiveness and efficiency in humanitarian action. It explores the methodological options and challenges associated with generating high quality evidence needed to answer key questions about the performance of humanitarian assistance, including:

• whether assistance is reaching the right people, at the right time
• whether assistance is bringing about the desired changes in their lives (effectiveness)
• and whether assistance is being delivered in the right doses, ways and with manageable costs (efficiency).

The paper draws on six case studies and real-life examples from the academic literature in order to demonstrate how impact evaluation methods can be used successfully and in an ethical manner to improve humanitarian assistance.

The case studies include:

• Case study 1:  Multiple interventions or a multi-agency intervention
• Case study 2: Unanticipated emergencies
• Case study 3: A complex emergency involving flooding and conflict
• Case study 4: A protracted emergency – internally displaced peoples in DRC
• Case study 5: Using impact evaluations to estimate the effect of assistance after typhoons in the Philippines
• Case study 6: Using impact evaluations to estimate the effect of assistance in the recovery phase in the absence of ex ante planning

How have you used or intend on using this resource?

I have used this resource as reading material to learn about different methodological options for impact evaluation depending on the type of humanitarian context at hand. The paper usefully distinguishes between emergency, non-emergency and recovery and resilience contexts, and organises methods by these categories. It also provides specific case studies for even more specific contexts, as listed here. The final case study, looking at options in the absence of ex ante planning, was particularly interesting as it is a challenge commonly experienced by evaluators.

Why would you recommend it to other people?

This is an excellent, thorough resource that successfully details specific methods relevant for humanitarian settings. This is recommended for those looking for a more technical piece, especially evaluators themselves.