TRANSPARENT; RIGOUR; MULTIPLE; QUALITY
Accountability means demonstrating results communities, partners, funders and policy makers. R,M&E that is rigorous, transparent and relevant will produce evidence for accountability. In C4D our primary responsibility is to be listening to, learning from and reporting to community groups and partners. Achieving accountability depends on having clear and shared expectations about what is to be evaluated, what the evaluation questions are, and how you will go about answering them. Understanding who you are accountable to also requires clarity.
Where do we start?
Deciding on the purpose, and more specifically how the primary intended users of the RM&E intend to use the findings and what their expectations are, and how this balances with Learning-based purposes. Being clear about the purpose helps to guide decisions about quality standards, rigour, and reporting.
Incorporating and implementing accountability approaches in practice
Manage (and commission) an evaluation or evaluation system
Understand and engage stakeholders: As part of understanding and engaging stakeholders it can be useful think about accountability in a multi-dimensional sense, including accountability to donors (upward accountability and reporting), and accountability to colleagues, partners and collaborators and communities (horizontal accountability).
Define ethical and quality evaluation standards: The quality and ethical standards for C4D R,M&E should reflect the expectations of all the people and groups we are accountable to (donors and managers, partners and community groups). Defining and following quality and ethical standards is important for maintaining accountability and integrity in RM&E. Ensuring ethical practices in RM&E is a responsibility of everyone involved in the R,M&E.
Document management processes and agreements: Transparent and thorough record-keeping of management processes and agreements is supports accountability to all stakeholders in RM&E processes.
Identify potential unintended results: Unintended results can be both positive and negative. As part of being accountable it is important to minimise any harm from unintended results from C4D. We need to use tools to help us predict (as far as possible) unintended outcomes, together with monitoring processes to identify and respond to unpredictable and negative unintended impacts as quickly as possible.
Identify primary intended users: We often assume that the primary intended users of RM&E are the manager and donors. In C4D we to think more broadly about who might use the RM&E. Key users in C4D usually include community groups, partners and others with roles in planning and implementation. The C4D and the R,M&E should be accountable to all of these groups.
Deciding on the purpose: R,M&E can be useful for accountability purposes, because it can be used to report back to all people and groups connected to the C4D initiative (including donors, managers, partners, community groups, ‘beneficiaries’ and others).
Determine what 'success' looks like: Working with community groups, partners and others to find agreement about what success might look like means that everybody knows and understands what values are used to make judgements about a program. In other words, the criteria and values to judge success are shared and transparent.
|Describe (to answer descriptive questions)
Sample: Thoughtful and thorough sampling helps to make the R,M&E design more rigourous. In quantitative (numbers based) methods sampling the sample size and the sample selection are key to making credible claims about the findings. In qualitative (words, stories,visual) methods, sharing details about the sample and selection process increases credibility and trustworthiness.
Use measures, indicators or metrics: We usually think about indicators as being useful for reporting and accountability to managers and donors. Indicators should also be used for providing partners, communication groups and others participating in the intervention with information about what was achieved/not achieved, and the importance of the indicators for their community. When using the data from indicators in this way, it is important to acknowledge that the information is simplified and partial, and that other types of information are usually needed to make informed decisions about the intervention.
Combine qualitative and quantitative data: A key part of being accountable is rigour. Combining data from different data collection methods boosts the rigour by providing different perspectives and ways to understand a problem.
Analyse data: Those involved in data analysis are in a powerful position of meaning making. It is important that the processes used to analyse data are rigorous, systematic and transparent.
Manage Data: C4D emphasises good data management and ownership processes that are respectful, ethical, and responsible. It is important to agree to policies and processes that prevent or minimise harm (especially for vulnerable groups). These discussions should take place before, during and after the data collection.
|Understand causes (to answer questions about causes and contributions)
Investigate Causal Attribution and Contribution: A central question in RM&E from an accountability perspective is 'what has been the impact (or contribution) of C4D to observed changes'. Answering this question rigorously requires selecting carefully from three causal analysis strategies (Compare results to a counterfactual; Check the results support causal attribution; Investigate possible alternative explanations).
Synthesise Data from a Single Study/Evaluation: By undertaking data synthesis processes we can make findings based on different sources of evidence and voices. This is useful tool for accountability to partners and community groups, and to donors and managers.
|Report and support use||
Identifying Reporting Requirements: Reports from R,M&E are usually focused on satisfying the needs of donor and managers. These are important users, but it is also important to think about the reporting needs of other groups we are accountable to. This includes partners, community groups, local institutions and other stakeholders.
Developing recommendations: How can social accountability principles be used to ensure that recommendations from stakeholders are heard and meaningfully considered?
Supporting use: To achieve social accountability it is critical that recommendations from different stakeholders are heard and meaningfully considered by decision makers. Committed and transparent processes to ensure that the findings (both positive and negative) from R,M&E are used is an important part of accountability.
Challenges and strategies
|Systems like Results Based Management tend to prioritise reporting to managers and donors, rather than accountability to partners and community groups.||C4D can lead the charge in advocating for a multi-dimensional understanding accountability, as set out in the SDGs.|
In Results Based Management accountability is usually judged based on how well the initiative followed planned processes. This is not as suitable in complex situations or where experimentation and adaptability are key.
The following tasks and approaches can be useful to thinking about alternatives:
|Too much focus on ‘accountability’ for ‘results’ can make people fear failure. In C4D where outcomes are less controllable and predictable, teams might start limiting the objectives they set. This can also lead to less collaboration, where sections or organisations (in a joint project) only work on areas that they are accountable/responsible for.||The Theory of Change is a useful way to create a holistic overview of how we think different parts of the project contributions to change (see Develop program theory or logic model). Concepts like the sphere of control, sphere of influence, and sphere of concern, borrowed from Outcome Mapping (see Better Evaluation page about Outcome Mapping), can be useful for identifying the degree to which teams should be expected to be accountable for changes.|
|A common perception within some organisations is that high quality and credible evaluations require quantitative data.||
There are certainly ways to ensure rigour and quality in mixed-methods and qualitative designs. The following tasks can be useful:
A Toolkit for Monitoring and Evaluating Children's Participation
This toolkit provides guidance on how to monitor and evaluate (using participatory and inclusive methods) children's participation in programmes, communities, and in wider society. The quality and outcomes of participation is a notoriously difficult thing to evaluate. Booklet 3 is particularly useful from an accountability perspective, since it includes tools to measure the quality and outcomes of children's participation. Click here to read a summary of this resource, including an overview of the ways in which it is consistent with the C4D Evaluation Framework.
Measuring Empowerment? Ask Them - Quantifying qualitative outcomes from people's own analysis by Jupp, D. & Ali, S. I. (with contribution from C. Barahona). This case offers an alternative model for creating indicators. Participatory process were used to develop and measure progress against indicators of community empowerment. Accountability in this case is primarily maintained at the community level, but the data generated was repurposed for upward reporting requirements.
- Read more about it here, including the ways in which is is consistent with the C4D Evaluation Framework.
U-Report is an SMS-based, user-centered social monitoring tool which supports social accountability. U-Report supports social accountability by providing a platform on which citizens can voice concerns and priorities and provide feedback to governments, development agencies and other decision makers. Read more about U-Report and how it is consistent with the C4D Evaluation Framework.