Evaluation in contexts of fragility, conflict and violence: Guidance from global evaluation practitioners

This guide offers practical advice for designing, conducting, and using evaluations in fragile, conflict, and violence-affected contexts, covering, among other areas, methodology adaptation; remote data collection; evaluation reporting, dissemination and utilization.

This book, edited by Hassnain, Kelly, and Somma (2021), serves as a comprehensive guide for designing, managing, and conducting evaluations in fragile and conflict-affected contexts, addressing unique challenges that may impact the validity and outcomes of such evaluations. The authors emphasize the importance of mitigating these challenges, which may require redefining the evaluation’s direction, purpose, or scope. They also highlight the potential for evaluations to unintentionally exacerbate tensions if conflict sensitivity is not integrated into their design and approaches. The book is structured into three phases and seven steps to align with standard evaluation practices.

Key content

Key takeaways from the resource include:

Phase A: Designing the evaluation

1. Understand and adapt to the context

   • This step provides guidance on how to conduct analyses critical for tailoring evaluations to the unique challenges of fragile, conflict, and violence-affected contexts. It includes:

     • Conflict Analysis: Understanding the causes, dynamics, and actors in the conflict to ensure evaluations do not exacerbate tensions. This is vital as it helps avoid worsening the conflict and promotes peacebuilding.

     • Gender Analysis: Identifying different needs and risks for women, men, boys, girls, and gender minorities, ensuring gender-sensitive evaluations. This is essential because conflict and fragility often exacerbate gender inequalities.

     • Safeguarding and Protection Analysis: Protecting at-risk populations from harm or exploitation and ensuring ethical evaluation practices, crucial in contexts where vulnerability is heightened.

     • Stakeholder Analysis: Identifying all relevant stakeholders, their interests, and power dynamics to engage them effectively. This helps in addressing the diverse perspectives and needs within fragile settings.

2. Select evaluation methods and tools

   • This step provides guidance on choosing appropriate evaluation methods and tools that can handle the complexities of FCV contexts. It includes:

     • Reconstruct Theory of Change: Developing a clear understanding of how interventions are expected to lead to desired outcomes, which is particularly important in unpredictable environments.

     • Understand Evaluation Criteria: Using criteria such as relevance, effectiveness, efficiency, impact, and sustainability to ensure comprehensive evaluations.

     • Design Evaluation Questions: Formulating questions that address key issues and information needs, tailored to the specific challenges of FCV contexts.

     • Tailor Methods and Tools: Selecting and adapting methods and tools to fit the volatile and often dangerous environments of FCV settings.

3. Select the evaluation team

   • This step provides guidance on forming an evaluation team with the right mix of expertise and local knowledge, critical for navigating the risks and complexities of FCV contexts. It includes:

     • Choosing Team Members: Selecting individuals with relevant expertise and contextual knowledge to enhance the evaluation's effectiveness and safety.

     • Ensuring Team Safety: Implementing measures to protect the evaluation team, which is crucial in high-risk environments where the safety of evaluators can be a significant concern.

Phase B: Conducting the evaluation

4. Organize a field mission for data collection

   • This step provides guidance on planning and executing field missions to gather data essential for accurate evaluations in FCV contexts. It includes:

     • Gathering Primary Data: Collecting data directly from the field through surveys, interviews, and observations, crucial for obtaining first-hand insights in volatile settings.

     • Gathering Secondary Data: Using existing data sources to complement primary data collection, which helps fill gaps when direct access is limited.

5. Arrange remote data collection

   • This step provides guidance on using digital tools and innovative ICT techniques for remote data gathering, particularly relevant when field access is restricted. It includes:

     • Selecting Tools: Choosing appropriate digital tools for data collection in contexts where field missions are not feasible.

     • Tips for Remote Data Collection: Implementing best practices for remote monitoring and evaluation to ensure data accuracy and reliability in challenging environments.

6. Close the evaluation learning loop

   • This step provides guidance on ensuring continuous learning and adaptation throughout the evaluation, which is critical in dynamic FCV contexts. It includes:

     • Feedback Mechanisms: Establishing processes for feedback and reflection to improve evaluation practices and outcomes continuously.

     • Adaptation: Making real-time adjustments to evaluation methods and approaches based on emerging findings, which is essential in fluid and unpredictable environments.

Phase C: Using the evaluation

7. Report, disseminate, and use

   • This step provides guidance on effectively reporting and utilizing evaluation findings to maximize their impact in FCV contexts. It includes:

     • Reporting: Developing clear, concise, and actionable reports that communicate findings to stakeholders, ensuring they are understandable and useful.

     • Dissemination: Using various channels to share evaluation results widely, ensuring accessibility for different audiences, which is crucial for promoting transparency and accountability.

     • Utilization: Promoting the use of evaluation findings to inform decision-making, improve interventions, and enhance organizational learning, which is vital for achieving sustainable impact in FCV settings.