Define what is to be evaluated in FCV settings

In fragile, conflict-affected, and violent (FCV) settings, the task of defining what is to be evaluated is both crucial and complex.

This process involves understanding the situation, developing a theory of change, and identifying potential unintended consequences.

On this page

Overarching principles

Particularly relevant principles for defining the focus of M&E activities include:

  • Conduct regular situation analysis: Regular situation analysis is essential for understanding the situation. In FCV contexts, conditions can change rapidly. Continuous assessment ensures that M&E remains relevant and accurately reflects the current environment.
  • Respect local contexts and involve communities: Engaging local stakeholders enhances understanding of the context by incorporating diverse perspectives and local knowledge. This is important for developing a theory of change that aligns with local realities and for identifying potential unintended consequences that might not be apparent without community input.
  • Do no harm and ensure conflict sensitivity: By integrating a conflict-sensitive and ‘do-no-harm’ approach into the development of a ToC, M&E practitioners can explore how interventions might interact with existing conflicts, allowing them to adjust strategies to avoid exacerbating tensions. This contributes to a ToC that is context-sensitive, ethically grounded, and proactively addresses potential unintended negative consequences.
  • Foster flexibility and adaptability: In FCV contexts, it's crucial to design theories of change (ToCs) that are flexible and adaptable due to the volatile and rapidly changing nature of these environments. By allowing for adjustments as new information emerges or circumstances evolve, evaluators ensure that interventions remain relevant and effective. This responsiveness helps address unforeseen challenges and capitalize on emerging opportunities within the dynamic environment.
  • Ensure psychological safety and trauma-informed practice: When focusing on M&E processes in FCV contexts, ensuring psychological safety and adopting trauma-informed practices are essential when defining what is to be evaluated. This means that during the process of understanding the situation, developing theories of change, and identifying potential unintended consequences, evaluators should be sensitive to the psychological state and well-being of participants.

View the full list of overarching principles here.

Understand the situation

Understanding the complexities of FCV contexts is essential for planning, monitoring and evaluating interventions. These environments are shaped by interconnected drivers of fragility, sources of resilience, and risk factors (DRRs), which include socio-political, economic, and cultural elements, making it challenging to fully understand the situation.

A comprehensive understanding of the context can:

  • Inform development and revision of the ToC
  • Inform stakeholder engagement plans
  • Inform data collection strategies
  • Inform ongoing adaption to changing circumstances

Conduct risk assessments

Risk assessments are critical in FCV settings, where interventions often face volatile and rapidly changing conditions.

  • Risk assessments help to identify potential threats and proactively manage risks.
  • This includes evaluating safety concerns, political instability, potential harm to stakeholders, and the risk of exacerbating conflict.
  • Risk assessments should be conducted at the outset and revisited regularly as the context evolves.

Apply conflict scans and conflict analyses

Conflict scans and analyses are essential tools for understanding the drivers and dynamics of conflict in FCV contexts. These methods help evaluators and practitioners navigate complex environments and design interventions that are sensitive to local conflict dynamics.

  • Conflict scans provide a broad or rapid overview of the conflict landscape, identifying key actors, recent events, and potential triggers of violence.
  • Conflict analyses offer a more detailed exploration of the structural and proximate causes of conflict, examining factors such as historical grievances, socio-political dynamics, and the roles of stakeholders.

These methods help ensure that interventions are conflict-sensitive and adaptable to shifting conditions.

Conduct evaluability assessments

An evaluability assessment examines whether an intervention can be evaluated reliably and credibly.

An evaluability assessment assesses the intervention’s theory of change, the availability of relevant data, and stakeholder engagement to ensure that evaluation efforts will yield useful, actionable results.

Evaluability assessments are particularly important in FCV settings, where context and data availability can change rapidly.

Methods and approaches

A range of tools can be used to understand the drivers and contextual factors of an FCV setting. These include: 

  • Conflict analysis: A process that identifies and examines the root causes, actors, and dynamics of conflict within a specific context to inform conflict-sensitive interventions.
  • Evaluability assessments: A pre-evaluation process that determines whether a program or policy is ready to be evaluated and if meaningful evaluation results can be obtained.
  • Needs analysis: A systematic process to identify and assess the needs of a population or target group to inform the design and delivery of effective interventions.
  • Gender analysis: The study of gender differences and inequalities in a given context to understand how they affect and are affected by policies, programs, and interventions.
  • Stakeholder analysis: The identification and examination of all relevant parties affected by or involved in a project to understand their interests, influence, and potential impact.
  • Political economy analysis: A method of examining how political, economic, and social factors influence power dynamics, resource distribution, and decision-making in a specific context.
  • Situation analysis: An assessment that provides a comprehensive understanding of the current context, including environmental, social, economic, and political conditions.
  • Scenario planning / visioning: A strategic planning method that explores and prepares for multiple possible future scenarios to guide decision-making and long-term planning.
  • Methods for foresight: Techniques used to anticipate and prepare for future challenges and opportunities by identifying emerging trends, risks, and potential disruptions.
  • Triple A change space (used in PDIA): A framework in Problem-Driven Iterative Adaptation (PDIA) that emphasizes the importance of Authority, Acceptance, and Ability to drive sustainable change.
  • Thinking and working politically: An approach that integrates political context and dynamics into development work to effectively navigate and influence the political environment for sustainable outcomes.
    See also methods listed on the Rainbow Framework page Understand the situation.

Resources

Develop theory of change / programme theory

Theories of change (ToCs) can be used to map out and assess the assumptions and pathways through which interventions aim to achieve their goals. Rather than listing inputs, activities, outputs, and outcomes, ToCs should explain how activities are expected to contribute to a sequence of outcomes, recognising that other programmes and external factors will also influence results. They serve as a framework to guide data collection, explore causal pathways, and ensure that the evaluation is responsive to the dynamic nature of conflict-affected settings.

Include diverse views when developing TOCs

Where feasible and safe, involving a range of different stakeholder groups in developing and validating a TOC can improve its alignment with local realities and foster ownership.

  • Including diverse perspectives can help align the ToC with local realities.
  • Ensure participation does not endanger individuals, especially in fragile settings.
  • Use trauma-informed and sensitive processes when asking how change is achieved.

Ensure TOCs allow for flexibility and are relevant to the context

ToCs in FCV contexts should be adaptable, allowing for revisions as new information becomes available or as the context changes.

  • Regularly review and adapt ToCs as the context shifts to maintain relevance and effectiveness as situations and challenges evolve.
  • Drivers of Resilience and Risk Factors (DRRs) can also be included to explore how project activities lead to both short-term and long-term outcomes that mitigate specific FCV challenges.

Create visual, accessible versions

Visual representations of theories of change can help improve understanding and accessibility for all stakeholders.

  • Ensure accessibility: ToCs should be presented in a way that ensures everyone involved has a clear and shared understanding of the intervention's goals and impact.
  • Use visuals: Visual representations can enhance engagement across different stakeholder groups, including local communities, implementing partners, and donors.

Methods and approaches

Resources

Identify potential unintended results

Traditional M&E frameworks in unpredictable and complex settings often focus only on primarily on intended outcomes, potentially overlooking broader impacts that may unintentionally exacerbate tensions or cause harm in other ways.

Use theories of change to anticipate unintended results

ToCs can be a useful tool for identifying unintended consequences, both positive and negative. Rather than focusing solely on intended outcomes, incorporating potential unintended results into the ToC can help anticipate and address challenges or capitalise on unexpected opportunities.

  • Anticipate unintended results: Including both positive and negative potential outcomes in the ToC can make it a more effective tool for navigating complex environments.
  • Stay flexible during data collection: Being open to the possibility of unintended results as data is collected allows for a more flexible and adaptive evaluation process. This flexibility ensures that the ToC remains relevant as the intervention evolves.
  • Consider reconstructing the ToC if necessary: In cases where the original ToC is weak or absent, reconstructing it ex-post may help to identify both the risks and unintended effects of an intervention, especially in changing contexts (Hassnain, 2020).
  • Construct a negative theory of change: A "negative" theory of change involves identifying potential unintended consequences or failures and ensuring that strategies are in place to mitigate these risks. By anticipating these challenges in advance, a negative ToC helps to preemptively address potential risks and negative impacts, allowing for more proactive risk management.

Case study: Barrier analysis for unintended consequence identification in Burundi

Search for Common Ground is one of the largest INGO’s working explicitly on conflict transformation and has decades of experience in M&E in FCV. A 2017 article by members of the organization’s evaluation unit described their lessons on identifying unintended consequences across their operational settings. In one instance, evaluators used barrier analysis for this process:

“...conducting a barrier analysis at the beginning of a project on women’s political participation forced the Burundi team to question its assumptions and design the project around those needs from the beginning, and then evaluate based on the project’s responsiveness to those identified barriers, in addition to the goals and objectives of the logframe. Creative and participatory methodologies that allowed participants and community members to express what was important to them, unrelated to the project’s objectives, naturally guided evaluations towards a better examination of unintended effects.

Methods and processes

Some methods and processes that can be useful in identifying unintended results include:

  • Conflict sensitivity: Integrates local conflict dynamics into project stages to identify and mitigate potential negative impacts early, ensuring interventions do not worsen tensions.
  • Outcome Mapping: This method tracks changes in behaviours, relationships, and actions within a project's sphere of influence, helping to identify both intended and unintended effects in complex, non-linear environments, such as FCV settings.
  • Outcome Harvesting: A retrospective method that identifies unintended outcomes by working backward from observed results, useful in volatile FCV settings where outcomes may not align with predefined expectations.
  • Systems thinking: This approach can be used to map out the complex interrelationships and feedback loops in FCV contexts, revealing potential unintended consequences that might not be apparent through linear analyses.
  • Complexity-Aware Monitoring: Suited to dynamic FCV environments, this approach involves real-time data collection and adaptability, addressing uncertainties, unintended outcomes, and feedback loops to allow for more responsive and flexible M&E.
  • Key informant interviews: Involves interviews with individuals who have deep knowledge of the context, providing insights into unintended impacts that other data collection might miss in FCV settings.
  • Negative programme theory:
    Also called ‘dark logic models’, negative programme theory explores how interventions might produce negative outcomes, helping to anticipate and manage risks, which is especially useful in FCV contexts with unpredictable power dynamics.
  • Risk assessment: Identifies potential negative impacts and their likelihood, ensuring strategies are in place to mitigate risks, which is essential in FCV environments.
  • Barrier analysis: A rapid assessment tool used to identify the behavioural, cultural, and environmental barriers that prevent individuals or communities from adopting desired behaviours, which can help reveal unexpected challenges in FCV settings.
  • See more methods and processes on the Rainbow Framework task page: Identify potential unintended results.

Resources

Last updated:

'Define what is to be evaluated in FCV settings' is referenced in:

Last updated

Back to top