Describe activities, outcomes, impacts and contexts in FCV settings

This section covers key tasks related to data collection, management, and the use of indicators in fragile, conflict-affected, and violent (FCV) contexts, with a focus on prioritising safety, contextual relevance, and flexibility in response to the unique challenges of FCV environments.

On this page

Overarching principles

Principles that are particularly relevant for this cluster of tasks in FCV include:

  • Do no harm and ensure conflict sensitivity: Describing activities and outcomes must be done with a conflict-sensitive approach to avoid exacerbating tensions or putting individuals at risk. This includes anonymising sensitive data, using neutral language, and ensuring data collection and analysis activities do not inflame existing conflicts.
  • Ensure safety and security: Data collection and descriptions must prioritise the safety of participants and evaluators. This includes using secure methods for collecting and managing data, anonymising personal information, and safeguarding participants against potential risks.
  • Foster flexibility and adaptability: Evaluators must be responsive to shifts in security, political, or social dynamics, ensuring that the data collected and analysis remain relevant, timely, and reflective of the current realities.
  • Build trust, transparency, and accountability: Transparency in how data is collected and activities are described helps build trust with stakeholders. Clear documentation of sampling decisions and methodological choices enhances the credibility of the evaluation.
  • Respect local contexts and involve communities in M&E processes and decision-making: Tailoring data collection methods and indicators to local contexts ensures that descriptions are relevant and culturally appropriate.
  • Empower and strengthen capacity in local communities: Describing outcomes and impacts should not only reflect the context but also help build local capacity.
  • Maintain and promote ethical standards: Ethical considerations are critical when collecting and describing data in FCV settings.

View the full list of overarching principles here.

Sample

In FCV settings, sampling methods must be flexible and adapted to the unique challenges of the environment.

Select appropriate and feasible sampling methods for the context

  • Some sampling methods that rely on complete data may not be feasible. Use available information and regularly update sampling frames to reflect population changes.
  • Consider using alternative data where comprehensive population data is unavailable. Sources from humanitarian agencies, NGOs, or community leaders can be used to construct sampling frames.

Prioritise safety and consider alternatives if there are security concerns

The Do No Harm principle must be applied throughout the sampling process to ensure participant confidentiality and safety.

  • Prioritise participant safety by avoiding methods that could expose individuals to risk.
  • Where security concerns limit travel or direct communication, consider alternative methods.
  • Conduct risk assessments to ensure data collection and management practices do not compromise participant anonymity or safety.

Document sampling decisions transparently

Transparency is essential to ensure the credibility of the sampling process and to contextualise findings for stakeholders (Bamberger et al., 2006).

  • Document the rationale behind the selected sampling methods, any adjustments made, and how these choices align with the evaluation’s goals.
  • Ensure that documentation does not compromise safety and maintains participant anonymity.

Resources

Use measures, indicators or metrics

When using measures, indicators, or metrics in FCV contexts, it's crucial to adapt these tools to the unique challenges and dynamic nature of such environments. Flexibility and context sensitivity are key to ensuring that the selected indicators remain relevant and meaningful throughout the intervention.

Ensure relevance of indicators

  • Contextualise indicators by ensuring they capture not only traditional outputs and outcomes but also the nuances of fragility, conflict, and violence. Integrate metrics related to peacebuilding, conflict sensitivity, and social cohesion alongside conventional indicators.
  • Consider data availability by selecting indicators for which reliable data sources exist or can be developed. In FCV contexts, data may be scarce or unreliable, so an important consideration for deciding on indicators is what data is accessible or feasible to collect.  Proxy indicators can provide indirect signals of outcomes, such as using market prices to reflect economic stability or school attendance to indicate levels of security. 
  • Select the most meaningful data type for each indicator rather than defaulting to quantitative or qualitative methods. Qualitative data, for example, might be more appropriate for capturing community perspectives on trust, security, or social cohesion, while quantitative measures may be useful for tracking more standardised outcomes.
  • Revisit and revise indicators regularly to ensure they stay relevant amidst shifting political, social, or security dynamics. Changes in activities, outputs, or the theory of change may require updates to the indicators to ensure they continue to reflect meaningful measures of success.

Choose between existing indicators and customisation

  • Use existing indicators for comparability and reliability where frameworks specific to FCV contexts are available. These can help ensure that the evaluation results are comparable to other interventions and align with widely accepted benchmarks.
  • Customise or develop new indicators when necessary to better reflect the unique and complex challenges of FCV environments. Existing indicators may not always capture critical aspects such as conflict sensitivity, social cohesion, or changes in community trust and security.
  • Involve stakeholders in the development process to ensure that indicators are both culturally and contextually appropriate, reflecting local realities and priorities in fragile and conflict-affected settings.
  • Conduct data rehearsals with intended users to verify that the selected indicators are relevant and useful for decision-making. This step ensures that the data being collected can inform adaptive management and real-time decisions in the dynamic environments typical of FCV contexts.

Methods and approaches

Different types of indicators:

In addition to indicators related to outputs, outcomes, and longer-term impacts, indicator types that are particularly relevant to FCV contexts include:

  • Process indicators capture information about whether interventions are being implemented as intended, which is critical for assessing the quality of execution. For example, a process indicator might track the number of program reviews conducted and whether lessons learned are being integrated into ongoing activities.
  • Sentinel indicators act as early warning signals in FCV contexts, helping to identify emerging risks like escalating conflict, deteriorating security, or impending humanitarian crises. For instance, a sudden increase in reports of violence or a rise in refugee movements could serve as sentinel indicators, prompting timely and adaptive responses.
  • Proxy indicators provide indirect measures of outcomes when direct data collection is unsafe or impractical. For example, market prices can serve as a proxy for economic stability, while school attendance might be used as a proxy indicator for security or social stability in conflict-affected areas.
  • Contextual indicators provide insights into the broader environment in which interventions take place, such as the political climate, security situation, or demographic changes. These indicators are essential for interpreting the relevance and impact of other evaluation findings.

Alternatives and complements to indicators

  • Rubrics: Rubrics can be particularly useful for capturing the complexity and nuance of performance in challenging environments where traditional indicators may fall short, especially when outcomes are difficult to quantify or the context is fluid. They allow for the integration of both quantitative and qualitative data and are especially valuable when different stakeholders have varying definitions of success. By defining clear criteria and levels of performance, rubrics make the evaluation process more transparent and inclusive.
  • Causal Link Monitoring: Causal Link Monitoring (CLM) complements traditional performance monitoring systems, which typically use fixed indicators to track activities and outcomes, by adding flexibility and adaptability to MEL systems. CLM focuses on uncertain and emergent aspects of a project. It allows teams to adjust monitoring efforts as the project evolves, ensuring that strategic points in the design are tracked and adapted based on real-time evidence. This approach enhances decision-making by incorporating both established indicators and dynamic observation areas.
  • Conflict sensitivity analyses and Political Economy Analysis (PEA): These tools can guide the selection and interpretation of contextual indicators that reflect the complex realities of FCV environments.

Resources

Collect and/or retrieve data

Data collection in FCV contexts is challenging for a number of reasons, including availability of data, restricted access, security risks, logistical issues, and cultural differences between evaluators and communities. It is essential to consider these challenges when designing data collection methods to ensure they are feasible, safe, and contextually appropriate.

Consider feasibility and utility of data collection methods

  • Assess data collection feasibility early by conducting pilots and addressing risks in inception reports. This allows for testing tools in the field and clarifying objectives with stakeholders. For example, a pilot could test whether remote sensing or phone interviews are feasible in areas with limited access.
  • Use data rehearsals to validate data utility and ensure that the data collected aligns with the needs of evaluation users. Data collection should be closely tied to the Theory of Change and Key Evaluation Questions, with an evaluation matrix helping to structure and guide the process effectively.

Engage M&E professionals who know the local context

  • Partner with local M&E professionals who understand the cultural and security context. This improves the quality of data collection and builds trust with local communities. For example, local data collectors may be better equipped to navigate cultural norms or security checkpoints.
  • Foster equitable partnerships by fairly compensating local evaluators, providing them with professional development opportunities, and involving them in decision-making processes from the outset.
  • Consider risks to local evaluators and data collectors and take steps to mitigate these, such as providing comprehensive security training, ensuring access to mental health support, and implementing robust safety protocols.
  • Remain flexible and prepared to switch to remote data collection methods like digital surveys, phone interviews, or remote sensing technologies when direct access becomes too risky.

Make context-appropriate decisions about methods

  • Prioritise Do No Harm and conflict sensitivity in method selection by considering how different tools and methods might affect the local context. Data collection methods should be evaluated not just for their effectiveness but also for how they might influence social dynamics, exacerbate existing conflicts, or introduce new risks.
  • Leverage digital tools cautiously by balancing their potential to offer innovative solutions with the challenges they pose. For example, digital tools like drone footage may be useful for monitoring land use changes but carry risks such as re-traumatising local populations or breaching privacy. Always assess the data security risks and impacts on communities.
  • Adopt a flexible and adaptive approach to data collection. Use a combination of methods and tools that can be adapted as the situation changes. Consider how traditional tools might be impractical in FCV settings due to security concerns, access limitations, resource limitations, or cultural barriers. Be prepared to adjust data collection strategies to align with shifting security conditions, access restrictions, or logistical challenges.
  • Tailor methods to cultural and power dynamics by selecting approaches that respect local norms and hierarchies. Individual interviews may be effective for gathering personal insights on sensitive topics, while focus groups might be more suitable for exploring collective experiences.
  • Adapt data collection formats as needed by, for example, switching from individual interviews to focus groups when power dynamics suggest it is more appropriate or hiring research assistants who share the gender or cultural background of respondents to foster openness.
  • Ensure informed consent is secured, ensuring participants fully understand the purpose of the evaluation and that their involvement is voluntary. This is critical where participation carries risks. Participants must know they can opt-out without negative consequences.

Methods and processes

Resources

Manage data

In fragile, conflict-affected, and violent (FCV) settings, managing data requires careful consideration of ethical, practical, and security challenges. The sensitive nature of information in these contexts makes robust data protection and responsible handling essential to safeguard participants and staff.

Mitigate risks associated with data breaches

  • Implement strong data protection measures by using encryption, secure storage, and password-protected devices with reliable antivirus software. Given the risks of data breaches, especially in FCV contexts, it's crucial to prevent exposure of personally identifiable information. Cloud-based systems may be useful for real-time data exchange and backup solutions, but they require thorough security and risk assessments to ensure they are safe to use.
  • Anonymise data before analysis and reporting, particularly for vulnerable populations. This reduces risks if data leaks occur, protecting participants from potential harm.
  • Reduce the risks associated with geographic data by using tools that conceal exact locations. This further protects sensitive information, especially when precise locations could endanger individuals or communities. 
  • Provide staff with training on ethical data management, including risk planning and responsive actions for unexpected challenges. This ensures they can respond effectively to challenges while safeguarding sensitive data.

Implement a coordinated data protection approach across partners

  • Establish clear data management protocols when working with third-party monitors or multiple actors. Ensure all partners, including local subcontractors, follow consistent data protection standards.
  • Include specific data handling clauses in contracts with third-party monitors, and conduct regular audits to ensure compliance with data protection practices.
  • Assess the risks and benefits of data sharing before disseminating information. While sharing data can enhance transparency and monitoring, always weigh these benefits against the potential risks to participants’ safety and privacy.
  • Implement protective measures for data sharing, such as like encryption and clear data-sharing guidelines to manage risks effectively. 

Methods, approaches:

  • Consistent data collection and recording

    An important aspect of data quality is to ensure data is collected consistently across different sites and different data collectors.

  • Data backup

    Data backup refers to onsite and offsite, automatic and manual processes to guard against the risk of data being lost or corrupted. 

  • Data cleaning

    Data cleaning involves the detection and removal (or correction) of errors and inconsistencies in a data set or database due to data corruption or inaccurate entry. 

  • Effective data transfer

    Effective data transfer involves processes to move data between systems, including between software packages, to avoid the need to rekey data. 

  • Secure data storage

    Processes to protect electronic and hard copy data in all forms, including questionnaires, interview tapes and electronic files from being accessed without authority or damaged.

  • Archive data for future use

    Putting systems in place to store de-identified data so that they can be accessed for verification purposes or for further analysis and research in the future, researchers can extend the range of the data collection efforts and encourage 

Resources

Last updated:

'Describe activities, outcomes, impacts and contexts in FCV settings' is referenced in:

Back to top