Manage an evaluation or M&E system in FCV settings
Successful management in fragile, conflict-affected, and violent (FCV) settings involves careful stakeholder engagement, strategic resource management, context-appropriate design, thorough documentation, and ongoing capacity strengthening to navigate the complexities of FCV contexts.
On this page
- Overarching principles
- Understand and engage stakeholders
- Decide who will conduct the evaluation
- Determine and secure resources
- Develop a design for the evaluation
- Determine what constitutes high-quality M&E
- Document management processes and agreements
- Develop planning documents for the evaluation or M&E system
- Review evaluation quality
- Strengthen evaluation capacity
Overarching principles
Principles that are particularly relevant for managing M&E activities in FCV include:
- Respect local contexts and involve communities in M&E processes and decision-making: Effectively understanding and engaging stakeholders is crucial for creating evaluations that are relevant and culturally appropriate. This involves actively involving community members in decision-making processes, developing communication and engagement plans for both during and after M&E activities and incorporating feedback into final decisions and outputs, ensuring that local perspectives shape the evaluation. Engaging stakeholders also helps navigate political challenges by managing conflicts among them and addressing limited government support, aligning evaluation objectives with diverse interests. However, it's essential to balance this inclusivity with efficiency, timeliness, and safety, especially in rapidly changing or violent environments, acknowledging the complex political challenges and adapting stakeholder engagement accordingly.
- Build trust, transparency, and accountability: Building trust, transparency, and accountability is essential in managing M&E activities because it fosters collaboration, secures stakeholder commitment, and enhances the credibility of the work. Engaging stakeholders early in processes like budgeting and planning demonstrates openness in resource allocation and builds trust. Documenting evaluation management processes and agreements and translating key documents into local languages ensures clarity among all stakeholders. Actively sharing findings with stakeholders and incorporating their feedback fosters accountability and strengthens relationships.
- Maintain and promote ethical standards: Maintaining and promoting ethical standards is essential in evaluations within fragile, conflict-affected, and violent (FCV) contexts because it ensures the credibility of the evaluation and the protection of all participants. Screening evaluation team members for potential biases and conflicts of interest is crucial. Ethical budgeting is also important, requiring sufficient allocation of resources for training on ethical practices, conflict sensitivity, and provisions for comprehensive ethical review processes. Implementing and regularly updating ethical guidelines tailored to FCV contexts guides decision-making and safeguards participant welfare throughout the evaluation. Emphasising secure data handling practices and participant protection is paramount, especially given the heightened risks present in FCV settings.
- Foster flexibility and adaptability: Flexibility and adaptability are vital in managing evaluations in FCV contexts due to unpredictable and changing environments. Adaptive management involves budgeting for adaptability to accommodate shifts in security, data collection methods, and ethical safeguards. Utilising adaptive budgeting techniques like flexible budgeting and scenario planning allows resources to adjust as needed in volatile environments. Employing flexible evaluation designs, like bricolage and iterative approaches, allows the evaluation to remain relevant as the context evolves. Combining structured plans with iterative adaptations balances clarity and responsiveness. Planning documents should allow for adjustments, with inception reports outlining how M&E activities might change in response to shifts.
- Ensure safety and security: Ensuring safety and security is a fundamental priority in FCV contexts to protect both evaluators and participants. Planning must account for safety measures when selecting teams and organising fieldwork, especially in high-risk areas. Providing training on conflict sensitivity, trauma-informed approaches, digital security, and field security prepares the team to navigate challenges safely. Including safety provisions in evaluation contracts further safeguards all involved. Depending on the security situation, adopting remote programming or methods that can be implemented remotely can help the evaluation to proceed while prioritising safety.
- Empower and strengthen capacity in local communities: Empowering local communities is essential for the sustainability and effectiveness of evaluations in fragile, conflict-affected, and violent (FCV) contexts. Involving local M&E professionals and community members who understand cultural nuances in M&E activities and decision-making can enhance the quality and relevance of the evaluation. Strengthening capacity through education, professional development, and on-the-job learning can help to enhance local expertise.
- Review quality and impact of M&E activities: Reviewing the quality and impact of M&E activities is crucial to ensure evaluations are effective, credible, and ethical in FCV settings. Implementing quality assurance mechanisms like advisory groups, ethical reviews, and peer reviews enhances credibility and helps uphold the "Do No Harm" principle by identifying and mitigating potential risks. Commissioning external meta-evaluations provides independent assessments that highlight areas for improvement. Engaging stakeholders through feedback loops keeps evaluations relevant and allows for adjustments based on input, reinforcing a commitment to ethical standards and the well-being of all participants.
- Act strategically to decide where to focus evaluation efforts: Acting strategically in focusing evaluation efforts maximizes impact and resource efficiency in FCV settings. Engaging stakeholders early in planning and budgeting can help to ensure resources are allocated appropriately, aligning efforts with pressing needs.
View the full list of overarching principles here.
Understand and engage stakeholders
Effective stakeholder engagement in FCV contexts is crucial to ensure that evaluations are sensitive to local dynamics, inclusive, and comprehensive. The complex political challenges highlighted in FCV settings make it even more important to adopt a strategic and flexible approach to stakeholder engagement. This is essential for navigating political challenges such as stakeholder conflicts and limited government support.
Consider how to involve community members throughout the evaluation process
Participatory approaches can be highly beneficial in FCV contexts, where conflicts among stakeholders and access restrictions are common.
- Actively involving key stakeholders and community groups throughout the evaluation process can not only enhance data collection and reduce bias but also ensure that community perspectives are well-integrated into the evaluation.
- These approaches can foster inclusivity and help navigate power dynamics by giving voice to less powerful stakeholders.
- In some cases, participation is used to improve data quality and interpretation, while in others, it is a means to support the agency by enabling stakeholders to actively shape the evaluation process itself.
Not all participatory M&E approaches are transformative by design. Some approaches may employ participatory and collaborative M&E methods for practical purposes without a strong commitment to shifting power dynamics. See Framing participatory evaluation for more insights about this distinction.
Balance inclusivity with efficiency, timeliness, conflict-sensitivity, and safety
M&E activities in FCV settings can often lack meaningful engagement with affected populations due to time and resource constraints. To address this, M&E commissioners and managers should consider:
- Allocating sufficient fieldwork time
- Hiring evaluators skilled in community engagement
- Prioritising affected populations' perspectives throughout
It’s also important to be aware that participatory approaches can put participants at risk from authorities or other groups. In some cases, it may be safer to avoid participatory methods altogether rather than implement them inadequately and endanger people.
- Assess potential risks to participants: Participatory methods may put individuals at risk from authorities or other groups.
- Decide when to avoid participatory methods: In some cases, it may be safer to use less direct engagement to protect participants.
Methods and approaches that are particularly useful:
- Identification and analysis of stakeholders: This involves identifying all relevant groups, including international aid agencies, civil society organisations, the private sector, government officials, opposition leaders, local community groups, armed actors, and beneficiaries.
- Background analysis: This analysis helps in understanding stakeholders' relationships to the intervention.
- Attitude and impact assessment: Can clarify how different groups view the project and its potential effects. Understanding these dynamics is vital for managing risks related to conflicts among stakeholders and aligning evaluation objectives with diverse interests.
- Stakeholder map: Visualizes stakeholders' power, interest, and influence to prioritise engagement strategies.
Resources:
- Guide to context analysis: Informing FAO decision-making – Approaches to working in fragile and conflict-affected contexts: Annex 2 of this guide focuses on stakeholder analysis, providing tools and methods for identifying key actors in fragile and conflict-affected contexts. It emphasizes the importance of understanding stakeholder interests, influence, and relationships to inform effective decision-making and programming within FAO’s work.
Decide who will conduct the evaluation
Human resource challenges, such as high turnover rates and the need for consultants, are common in FCV settings. Planning for these challenges, including budgeting for potentially higher costs associated with short-term specialists, can help maintain continuity and quality in M&E work.
Be aware of the potential for bias and conflicts of interest
It’s important to be aware that potential team members may hold certain biases and affiliations that can damage the credibility of the evaluation. Careful vetting helps ensure that the evaluation maintains its independence and ethical standing.
- Screen potential team members thoroughly: Examine their biases, affiliations, and perceptions of the conflict.
- Ensure neutrality: Confirm potential team members have no formal or informal ties to parties involved in the conflict to maintain the neutrality and credibility of the evaluation.
- Engage in open dialogues: Understand how potential team members interpret the conflict situation to avoid unintended biases.
The publication "Evaluation in Contexts of Fragility, Conflict and Violence" emphasizes the importance of unbiased assessment in sensitive environments. For more information, see the resource Evaluation in the Contexts of Fragility, Conflict, and Violence (PDF).
Value diverse skillsets
The evaluation team should be chosen not only for their technical capabilities but also for their ability to facilitate discussions, build trust among stakeholders, and navigate the complexities of working in sensitive environments.
- Including M&E staff from the local region who can communicate in the local language or dialect and bring insights into cultural nuances, community dynamics, and contextual understanding can further enhance the quality of an evaluation.
- Incorporating diverse skills and backgrounds into the evaluation team is crucial for addressing structural inequities and enhancing the cultural appropriateness and validity of the evaluation methods and findings.
Plan for logistical challenges
Logistical challenges, such as limited access to certain areas, high turnover of qualified professionals, and the need for specific security measures, further complicate the process of selecting an evaluation team.
- Strategic planning is essential to address these challenges (See Understand the context).
- Provide adequate training on conflict sensitivity and digital security, particularly given the heightened risks associated with data handling and participant protection in FCV settings.
Consider remote programming options if necessary
Depending on the security situation, remote programming is sometimes used by organisations and agencies who are not physically on the ground of implementation. There are various types of remote programming with different levels of collaboration and local leadership:
- Remote control: Typically implemented as a reactive measure when security conditions deteriorate, this approach involves minimal delegation of authority to local or field staff. The focus is on maintaining control from a distance and decision-making remains centralised with the external organisation. The approach is often a last resort when other options are not viable and typically has little emphasis on capacity development or skills transfer.
- Remote management: Also typically reactive, but with a greater degree of delegation, this modality involves external organisations working with local implementers. While some authority is transferred, the primary control still resides with the external staff, who often intend to regain full control when security improves. There is often moderate investment in capacity building, and procedures are typically established to enhance monitoring and quality assurance.
- Remote support: A more proactive strategy where external organisations actively invest in building the capacity of local staff, with the intention of eventually transferring full authority to them. This approach involves planning for the long term, including mentoring and developing the skills needed for national teams to take on leadership roles. The focus is on sustainability and empowering local actors to lead.
- Remote partnership: A proactive strategy where equal partnership is fostered with a local partner that already has significant internal capacity. The external organisation supports the local partner with resources, administrative support, and advocacy efforts, while the local partner leads on-ground implementation, leveraging their contextual knowledge and relationships.
Methods and approaches:
- Participatory evaluation: This is a family of collaborative methodologies that include programme participants and community members in many or all stages of evaluations. This approach is particularly valuable in fragile and conflict-affected settings as it can build trust, promote local ownership, and ensure that evaluations are culturally sensitive and relevant to the specific context, which is crucial in volatile environments where external perspectives may be limited or biased.
- Horizontal evaluation: This structured participatory workshop-based approach involves self-assessment from community members with external review by peers. In FCV, this method can be well suited because it combines local knowledge with external expertise, fostering mutual learning and reducing power imbalances that can exacerbate tensions in sensitive environments.
- Empowerment evaluation: A stakeholder involvement approach designed to provide groups with the tools and knowledge they need to monitor and evaluate their own performance and accomplish their goals. This method is well-suited for FCV settings as it builds local capacity, enhances self-reliance, and promotes sustainable development practices, which are essential for long-term stability and resilience in challenging contexts.
- Competencies: Refer to the knowledge, skills, attitudes, and behaviours needed for various roles within the evaluation process for high-quality evaluation, which may include organizational, sector, and region-specific competencies. Developing these competencies is crucial in FCV settings as it enables local actors to conduct culturally appropriate and conflict-sensitive evaluations, ensuring that M&E practices do not inadvertently exacerbate tensions or compromise the safety of participants.
- Everyday Peace Indicator Approach: This is a type of participatory methodology that draws on participatory numbers specifically for FCV settings where communities establish and prioritize indicators for design, monitoring, evaluation and research in their own communities by various actors.
Resources:
- Equitable Evaluation in Remote and Sensitive Spaces: This resource provides an example (Case Example 2) of prioritising programme participants in the selection of evaluators and methods to ensure fairness and relevance in remote and sensitive contexts.
- Evaluation in Contexts of Fragility, Conflict and Violence: This resource offers guidance on selecting evaluation teams or partners, with a focus on considerations like contextual expertise.
- Conflict Analysis: Section 3.2 of this guide discusses team composition, emphasising the importance of including diverse skill sets and perspectives to ensure a comprehensive understanding of the conflict.
- How to Guide to Conflict Sensitivity: Section 1.2 offers advice on selecting individuals to conduct conflict analysis.
- Sowing & Harvesting: Stage 1 of this handbook outlines how to establish an evaluation team, providing guidance on the composition and roles necessary for conducting participatory evaluations.
- Technology for Evaluation in Fragile and Conflict Affected States: An Introduction for the Digital Immigrant Evaluator: This resource discusses the skills required by evaluation teams to effectively utilise technology when conducting evaluations in fragile and conflict-affected states, focusing on digital competencies.
- Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: This resource provides guidance on selecting an evaluation team for peacebuilding initiatives in conflict and fragile settings, highlighting the need for conflict sensitivity and peacebuilding experience.
- Evaluation of Humanitarian Action Guide: This guide includes practical advice on contracting evaluators for humanitarian action.
Determine and secure resources
Developing a stand-alone, realistic M&E budget that adequately addresses ethical considerations and includes detailed allocations data collection, human resources, training, and logistics, along with a contingency for unforeseen challenges, is essential.
-
Budgeting for adaptive management is important: Adaptive management and budgeting (see suggested methods of alternative budgeting below) can help ensure enough flexibility to accommodate shifts in the security situation, changes in data collection methodologies, and the need for additional ethical safeguards, such as participant protection and secure data handling practices.
-
Secure support for resources: Engaging stakeholders early in the budgeting process is critical to securing their commitment and ensuring that resources for M&E are not reallocated to other project areas, which is a common issue in FCV settings where immediate needs often overshadow the long-term importance of evaluation and learning.
-
Include provisions for ethical processes and training: It’s vital that M&E budgets are sufficient to ensure evaluations adhere to ethical principles. This includes budgeting for training for team members on ethical practices, conflict sensitivity, and Do No Harm (DNH) approaches and provisions for comprehensive ethical review processes.
Methods and approaches:
- Rapid response funds: This is a type of funding mechanism that organizations may establish and employ to provide grants for quick response to disaster and emergency events.
- Pools of funding for specific purposes, like specialised expertise (PACT)
- Costed scenario planning: This method entails anticipating and estimating expected and potentially unexpected costs related with conducting the evaluation, especially in volatile and dynamic settings.
- Flexible budgeting: Adjusts based on actual activity levels to better align costs with performance.
- Rolling budgets: Continuously updated to reflect changes in the business environment.
- Zero-based budgeting (ZBB): Justifies all expenses from a "zero base" each new period.
- Beyond budgeting: Replaces traditional budgeting with adaptive management processes.
- Agile budgeting: Applies agile principles for frequent budget adjustments.
- Participatory budgeting: Engages stakeholders in deciding public budget allocations.
- Dynamic resource allocation: Continuously reallocates resources based on shifting priorities.
- Scenario planning: Uses strategic planning to prepare budgets for multiple potential futures.
Resources:
- How to Improve Results in Situations of Fragility, Conflict and Violence: 12 Recommendations: Recommendation #8 of this resource advises budgeting more purposefully for monitoring and evaluation (M&E) in FCV settings.
- Defining the Agenda: Key Lessons for Funders and Commissioners of Ethical Research in Fragile and Conflict-Affected Contexts: This resource offers mitigation strategies to address ethical issues, including guidance on fairly allocating resources between local and external researchers when conducting research in fragile and conflict-affected contexts.
- Technology for Evaluation in Fragile and Conflict Affected States: Sections 5.1 and 4.1.c discuss the costs of technology for evaluations, including procurement options and considerations for budgeting in fragile and conflict-affected states.
- Ethical Research in Fragile and Conflict-Affected Contexts: Guidelines for Applicants: Criteria 7 of this resource highlights the importance of flexible, fair, and transparent budgets and timelines that account for the complex needs of ethical research in fragile and conflict-affected contexts.
- Conflict Analysis: This section provides guidance on commissioning conflict analysis, emphasising the need for appropriate budgeting and planning to ensure thorough and context-sensitive analysis.
- How to Guide to Conflict Sensitivity:
- This section offers guidance on incorporating conflict sensitivity into project budgets, ensuring that financial planning accounts for the potential impact of conflict on project activities.
- Reflections on Applying Principles of Equitable Evaluation: Cost planning: This resource discusses the application of equitable evaluation principles, including cost planning that ensures fairness and inclusivity in the budgeting process.
- Technologies for Monitoring in Insecure Environments: This resource includes budget calculation tables and cost considerations for using mobile phones, digital data entry devices, remote sensing, location tracking, radio programmes, and online platforms in insecure environments.
- The Use of Third-Party Monitoring in Insecure Contexts: Section 4.3 and Annex 1 of this resource discuss the required investments of time and resources for third-party monitoring, including budget considerations for insecure contexts.
- Monitoring and Evaluation: Five Reality Checks for Adaptive Management: This resource provides practical insights on budgeting and resource allocation in the context of adaptive management, helping to align M&E practices with real-world constraints and changing conditions.
Develop a design for the evaluation
In FCV contexts, the choice of who leads the evaluation design should be guided by the need for contextual relevance, stakeholder engagement, and ethical considerations, with a preference for approaches that incorporate local knowledge and foster collaboration.
Consider who should be involved in the evaluation design
- Joint evaluation designs that involve multiple stakeholders (e.g., implementing agencies, funders, community members, local NGOs, or other in-country partners) can help balance different perspectives and ensure that the evaluation design is comprehensive and contextually appropriate. They are often the most effective, combining the strengths of all involved parties to create a contextually appropriate, ethical, and widely accepted evaluation.
- Ethical considerations and resource constraints are critical factors, and awareness of power dynamics is essential to ensure an inclusive and sensitive evaluation process.
Whether the design is led by commissioners, evaluators, or the community, it should prioritise conflict sensitivity and practicality.
Ensure the design is feasible and appropriate for the context
The nature of the evaluation should guide the design to ensure it effectively answers different types of key evaluation questions (KEQs).
- The design must address the specific challenges and complexities of FCV contexts, considering the intervention's complexity and the environment.
- Tailored approaches in data collection and analysis are necessary to manage these complications effectively, ensuring that the evaluation remains relevant, comprehensive, and sensitive to the context.
- The design will also need to align with the available resources, including time, budget, and existing data, and it’s important to be pragmatic in selecting methods that are feasible within the constraints of the FCV context.
Consider designs that can be adapted to the context and changes
A bricolage evaluation design is often recommended due to its flexibility in combining and adapting various data collection and analysis methods, approaches, and conceptual frameworks. This approach is particularly effective in addressing the unique and dynamic challenges of FCV settings, where adaptability and responsiveness are crucial.
Iterative designs offer the flexibility to make ongoing adjustments, making it ideal for the evolving nature of FCV contexts. While an upfront design can provide clear structure and direction—valuable in more stable settings—it may become too rigid if conditions change rapidly.
Methods and approaches:
- Joint evaluation design: a collaborative approach to evaluation design with the evaluation commissioner, implementing agency, program participants, and community members.
- Complexity: an overarching approach to viewing interventions, M&E activities, and the context in which they operate, informed by complexity science and systems thinking.
- Iterative design: a process for establishing an initial evaluation design at the outset of the evaluation and updating and refining the design as initial findings and developments emerge.
- Upfront design: a traditional process of establishing an evaluation design by the commissioner or evaluation team that is held constant throughout the evaluation and updated based on specific conditions outlined in the terms of reference.
- Bricolage design: an approach for customising evaluation designs with an eclectic mix of approaches, designs, and methods drawn from various evaluation models and procedures to suit the demands and needs of the evaluation context.
Resources:
- Impact Evaluation in Settings of Fragility and Humanitarian Emergency:
- Guidelines to Conflict Sensitive Research
- Evaluation of Humanitarian Action Guide
- How to Improve Results in Situations of Fragility, Conflict, and Violence: 12 Recommendations
- Evaluating Peacebuilding Activities in Settings of Conflict and Fragility
- Ethical Research Landscapes in Fragile and Conflict-Affected Contexts: Understanding the Challenges
- Equitable Evaluation in Remote and Sensitive Spaces
Determine what constitutes high-quality M&E
Evaluation quality can relate to products such as reports and communication outputs or planning or management documents, or processes in terms of how the evaluation was carried out. Different criteria for assessing quality of both M&E products and processes are listed on the task page Determine what constitutes high-quality evaluation.
Some particular considerations for FCV contexts include:
- Ethical practice: High-quality evaluation in FCV contexts must have ethical practice at its centre. Ethical standards must be rigorously maintained and adapted throughout the project lifecycle.
- Bias reduction: Bias reduction involves identifying possible sources of bias and taking steps to reduce it. This is one way of improving the validity of an evaluation. Cognitive biases must be minimized by incorporating diverse perspectives and data sources, ensuring that data interpretation is balanced and contextually accurate.
- Inclusion of diverse perspectives: Inclusion of diverse perspectives requires attention to ensure that marginalised people and communities are adequately engaged in the evaluation.
- Rigour: Rigour involves using systematic, transparent processes to produce valid findings and conclusions. However, there are varying notions of what constitutes rigour, especially in how causality is understood and demonstrated. In FCV contexts, rigour requires contextually appropriate approaches that can better and more appropriately explore causality within the complexity and unpredictability of these environments. See "Rethinking rigour to embrace complexity in peacebuilding evaluation " for an in-depth treatment on re-thinking ‘rigour’ in FCV settings.
- Common good and equity: Consideration of common good and equity involves an evaluation going beyond using only the values of evaluation stakeholders to develop an evaluative framework to also consider common good and equity more broadly. The Equitable Evaluation Framework (EEF) provides a useful foundation for defining success in FCV evaluations, shifting from traditional metrics to an equity-focused, participant-centered approach. Its key considerations include: diversity of evaluation teams, cultural appropriateness, uncovering inequities, and community empowerment.
- Utility: Utility is crucial in evaluations in FCV contexts because it ensures that the findings are actionable and directly contribute to improving program effectiveness in these challenging environments. Prioritising local knowledge and integrating Monitoring, Evaluation, and Learning (MEL) with program activities can help evaluations remain relevant and responsive to the immediate needs of affected communities.
Resources
- Generally accepted meta-evaluative frameworks: many of the above criteria for quality M&E practice can be found in standards of evaluation practice such as the transdisciplinary Program Evaluation Standards, the organizational UN Norms & Standards for Evaluation, or the regionally-specific African Evaluation Association principles.
Document management processes and agreements
Documenting evaluation management processes and agreements in FCV contexts is challenging due to the need for flexibility in unpredictable environments.
Adaptive management is essential, as conditions can change rapidly. Therefore, management documents, such as Terms of References (ToRs), should be designed to allow for adjustments without compromising the evaluation’s credibility or objectives.
A process should be put in place for documenting changes and adaptations, as well as processes for learning and adaptation, such as Learning Review Meetings. Clarity and transparency are also crucial. Roles and responsibilities should be clearly defined, even when they overlap in participatory approaches. Translating key documents into local languages helps ensure all stakeholders understand the evaluation’s goals and their roles, allowing them to identify potential risks.
Evaluation contracts should include provisions for safety and security, such as field security training, to protect both evaluators and participants. Management documents should be informed by a thorough situation analysis.
Methods and approaches
- Learning Review Meetings: These are regular moments to reflect on processes and stages of the evaluation, such as after action reviews.
- Situation Analysis: these are methods for assessing the situation and factors that contribute to observed phenomena.
Resources
- Conflict Sensitivity Tools and Guidance: This resource provides a conflict sensitivity checklist (Tool 3) for assessing programme documents and partner proposals to ensure they account for conflict-related risks and sensitivities.
- Monitoring, Evaluation and Learning for Fragile States and Peacebuilding Programs: Tool 21 and Tool 22 of this resource offer sample ToRs for mid-term and final evaluations, tailored to fragile states and peacebuilding programs.
- Evaluating Peacebuilding Activities in Settings of Conflict and Fragility: ANNEX C of this resource contains a sample Terms of Reference (TOR) specifically designed for conducting conflict evaluations in fragile and conflict-affected settings.
Develop planning documents for the evaluation or M&E system
In FCV contexts, evaluation planning documents must be flexible, context-sensitive, and ethically sound.
- Evaluation plans and frameworks should allow for iterative adjustments to adapt to rapidly changing conditions.
- Inception reports should acknowledge and reflect on the rapidly changing and unpredictable nature of the environment, and outline how the evaluation might need to change in response to shifts in the local situation, such as new security concerns, political developments, or other unforeseen challenges.
- Evaluation work plans need to be realistic, with timelines aligned to local contexts, such as agricultural cycles, and include contingency plans for disruptions. Emphasising local knowledge integration and ensuring the utility of findings are crucial.
- Data collection protocols must prioritize security and ethical considerations, with sensitive information handled carefully in summaries like aide-memoires.
Methods and approaches
Resources
- Conflict Sensitivity Tools and Guidance:This resource provides a checklist (Tool 5) for ensuring conflict sensitivity within monitoring and evaluation plans, helping to identify and mitigate conflict-related risks during M&E processes.
- How To Guide to Conflict Sensitivity: Section 2 of this resource discusses how to integrate conflict sensitivity throughout the programme cycle, offering guidance on ensuring that programmes do not inadvertently exacerbate conflict.
- Evaluation of Humanitarian Action Guide: This resource includes an example of an inception report, which outlines key elements such as evaluation objectives, methodologies, and timelines tailored to humanitarian action contexts.
- Monitoring, Evaluation and Learning for Fragile States and Peacebuilding Programs: Tool 20 of this resource provides an Evaluation Planning Worksheet to assist in designing and planning evaluations in fragile states and peacebuilding contexts.
Review evaluation quality
In fragile and conflict-affected settings, ensuring the quality of evaluations is crucial – both to ensure limited resources are used effectively, as well as to ensure monitoring and evaluation activities meet ethical standards.
Methods and approaches
The processes and tools listed below can be used to review and improve evaluation quality throughout its lifecycle:
- Advisory group: Establish an advisory group to provide ongoing guidance and feedback on the evaluation, ensuring that it remains contextually appropriate and ethically sound.
- Guidelines and ethical protocols: Implement and regularly update ethical guidelines tailored to the specific challenges of FCV contexts to guide decision-making and protect participant welfare. These should incorporate conflict sensitivity and focus on providing guidelines to ensure participant privacy and data confidentiality, and avoiding exacerbating conflicts or inequalities, especially when employing local evaluators.
- Ethical review processes: Implement ethical review mechanisms, particularly for technological tools, to protect participant rights and prevent worsening conflicts.
- Engage stakeholders: Establish feedback loops with participants and community members to validate findings and manage dissemination risks, thus enhancing the relevance and acceptance of the evaluation outcomes (Sources: Doing Research in Conflict Contexts; Guidelines to Conflict Sensitive Research).
- Implement peer review and feedback mechanisms: Engage with other specialists and stakeholders for feedback on the evaluation report. Conduct peer reviews with evaluation experts to enhance the quality and credibility of the report. Implement feedback loops with respondents to validate and refine findings before publication (Bentele, 2020).
- Commission external meta-evaluation: Hire third-parties to conduct external meta-evaluation at various points to gain an independent assessment of your evaluation processes. This can provide credibility, identify areas for improvement, and validate the effectiveness of your methods, ultimately enhancing the overall quality and reliability of your M&E activities.
Resources
- Assessing the Quality of Humanitarian Evaluations: The ALNAP Quality Proforma (2005): This resource provides a structured tool (the ALNAP Quality Proforma) to assess the quality of humanitarian evaluations. The Proforma offers a checklist that evaluates key aspects such as methodological rigor, clarity of findings, and utility for decision-making. It is particularly useful for practitioners seeking to ensure that evaluations in humanitarian settings meet high standards of quality and accountability.
- Strengthening the Quality of Evidence in Humanitarian Evaluations
This resource offers guidance on improving the quality of evidence in humanitarian evaluations, focusing on principles and approaches to ensure credible, useful findings. It provides practical insights for evaluators working in complex humanitarian environments.
Strengthen evaluation capacity
One of the primary challenges in FCV settings is the lack of institutional infrastructure and human resource capacity, a consequence of prolonged conflict and instability. Strengthening evaluation capacity in contexts of fragility, conflict, and violence (FCV) requires a multi-level approach, focusing on individual, organizational, and national capacities.
Individual capacity
The disruption of education systems and the relocation of professionals due to conflict often result in a shortage of local M&E expertise.
Targeted education and professional development programs can be a useful part of this. Partnerships between local universities and international institutions can help create M&E-focused curricula, while scholarships and training programs build a sustainable pool of local evaluators.
Integrating digital tools and the Internet offers opportunities for capacity building through interactive online training, guidance resources, and online communities of practice. These tools provide 'just-in-time' support and ongoing capacity strengthening, but their adoption can be made difficult by budgetary constraints, infrastructure limitations, and organizational resistance.
Individual capacity assessments against relevant competencies are a useful tool to guide capacity development plans.
On-the-job learning is another important avenue for strengthening individual capacity. There are a number of formal and informal ways of supporting individuals to learn while doing, including mentoring, peer review, self-reflection exercises. More information on different options for this can be found here.
Organisational capacity
Embedding evaluation practices within institutions, such as government ministries and NGOs, helps reduce dependency on external evaluators. Strengthening the evaluation culture of an organisation is critical. Establishing evaluation units equipped with the necessary resources and authority to carry out evaluations independently is one way to help foster a culture of accountability and evidence-based decision-making. Strategies and processes to support the use of M&E information are also important.
Strengthening organisational capacity in FCV contexts requires integrating both conflict sensitivity and ethical considerations into the core of the organisational culture. This involves comprehensive training for all staff on the principles of conflict sensitivity, ensuring that these principles guide every aspect of M&E work to avoid exacerbating conflicts or creating new tensions. Additionally, it is crucial to develop a deep understanding of the specific ethical challenges in FCV settings. This includes providing training on managing data, including informed consent, data protection, and the ethical implications of data sharing. Organisations should establish and enforce clear internal guidelines, ensuring that staff are fully equipped to navigate the complex and sensitive ethical landscape of FCV contexts.
Strengthening an organisation's capacity to adapt to rapidly changing and unpredictable environments is crucial in FCV settings. This includes building the skills and knowledge within the organisation to modify M&E strategies in response to sudden changes, such as shifts in conflict dynamics or humanitarian needs. Capacity strengthening should focus on training staff to be flexible and responsive in their approach and on putting contingency plans into action, ensuring that M&E activities can continue effectively even under challenging circumstances.
To ensure that capacity strengthening is sustainable, organisations should institutionalise these practices by embedding them into organisational policies and procedures. This can include creating dedicated roles or units focused on capacity building, or incorporating this function into dedicated M&E units, integrating capacity strengthening goals into project planning and evaluation, and continuously monitoring and evaluating the effectiveness of capacity building efforts.
National capacity
Building strong networks of collaboration between international organisations, local professionals, and other stakeholders is key to capacity strengthening. Joint evaluations and collaborative projects can help transfer knowledge and build capacity within local organisations. Creating forums for sharing context-specific experiences and best practices can also help build a collective knowledge base that supports capacity strengthening across different organisations. VOPES and evaluation societies can play an important role in creating networks and sharing opportunities.
For more information on national M&E capacity in FCV contexts, see the earlier section on national capacity.
Methods and approaches
- Enabling environment: This section of BetterEvaluation’s Strengthening National M&E Systems framework has information on the legal, institutional, political, and cultural factors that create a supportive environment for M&E practices, focusing on the legal frameworks and the culture of evidence use within government entities.
- Organisational capacity: This section of BetterEvaluation’s Strengthening National M&E Systems framework has information on strengthening organisational capacities in M&E, focusing on policies, data systems, quality assurance, and the use of evidence in decision-making.
- Interventions and tools for strengthening M&E capacity: This section of BetterEvaluation’s Strengthening National M&E Systems framework has information on various interventions and tools for strengthening M&E capacity, including diagnostic assessments, capacity-strengthening plans, technical assistance, formal education, other learning methods, knowledge sharing, and recognising good practices.
- Strengthen evaluation capacity: This task in the BetterEvaluation Rainbow Framework contains details on various methods for strengthening evaluation capacity, particularly at the individual level, including competency assessments, coaching, mentoring, peer learning, self-paced learning, professional development courses, and reflective practice, aimed at building knowledge, skills, and evaluative thinking.
- Competencies: This page provides links to a number of different M&E competency frameworks.
Resources
- MESA: GEI's Diagnostic Tool for a Monitoring and Evaluation Systems Analysis
- Solomon Island MESA diagnostic report: The Solomon Islands MESA Report explores the evaluation capacity in the Solomon Islands, highlighting the challenges and opportunities for strengthening M&E practices in the region.
- Building Statistical Capacity in Fragile and Conflict-Affected States
- Addressing the Lack of Evaluation Capacity in Post-Conflict Somalia: Table 4. A Checklist for Building Organizational Evaluation Capacity in Post-Conflict Somalia
- How to Improve Results in Situations of Fragility, Conflict and Violence: 12 Recommendations: Recommendation #7: Strengthen in-country M&E capacity and systems
Expandir para ver todos los recursos relacionados con 'Manage an evaluation or M&E system in FCV settings'
'Manage an evaluation or M&E system in FCV settings' is referenced in:
Theme