Transparent; rigour; multiple; quality
Accountability means demonstrating results communities, partners, funders and policy makers. R,M&E that is rigorous, transparent and relevant will produce evidence for accountability. In C4D our primary responsibility is to be listening to, learning from and reporting to community groups and partners. Achieving accountability depends on having clear and shared expectations about what is to be evaluated, what the evaluation questions are, and how you will go about answering them. Understanding who you are accountable to also requires clarity.
Where do we start?
Deciding on the purpose, and more specifically how the primary intended users of the RM&E intend to use the findings and what their expectations are, and how this balances with Learning-based purposes. Being clear about the purpose helps to guide decisions about quality standards, rigour, and reporting.
Incorporating and implementing accountability approaches in practice
Manage (and commission) an evaluation or evaluation system
Understand and engage stakeholders: As part of understanding and engaging stakeholders it can be useful think about accountability in a multi-dimensional sense, including accountability to donors (upward accountability and reporting), and accountability to colleagues, partners and collaborators and communities (horizontal accountability).
Define ethical and quality evaluation standards: The quality and ethical standards for C4D R,M&E should reflect the expectations of all the people and groups we are accountable to (donors and managers, partners and community groups). Defining and following quality and ethical standards is important for maintaining accountability and integrity in RM&E. Ensuring ethical practices in RM&E is a responsibility of everyone involved in the R,M&E.
Document management processes and agreements: Transparent and thorough record-keeping of management processes and agreements is supports accountability to all stakeholders in RM&E processes.
Identify potential unintended results: Unintended results can be both positive and negative. As part of being accountable it is important to minimise any harm from unintended results from C4D. We need to use tools to help us predict (as far as possible) unintended outcomes, together with monitoring processes to identify and respond to unpredictable and negative unintended impacts as quickly as possible.
Identify primary intended users: We often assume that the primary intended users of RM&E are the manager and donors. In C4D we to think more broadly about who might use the RM&E. Key users in C4D usually include community groups, partners and others with roles in planning and implementation. The C4D and the R,M&E should be accountable to all of these groups.
Deciding on the purpose: R,M&E can be useful for accountability purposes, because it can be used to report back to all people and groups connected to the C4D initiative (including donors, managers, partners, community groups, ‘beneficiaries’ and others).
Determine what 'success' looks like: Working with community groups, partners and others to find agreement about what success might look like means that everybody knows and understands what values are used to make judgements about a program. In other words, the criteria and values to judge success are shared and transparent.
Describe (to answer descriptive questions)
Sample: Thoughtful and thorough sampling helps to make the R,M&E design more rigourous. In quantitative (numbers based) methods sampling the sample size and the sample selection are key to making credible claims about the findings. In qualitative (words, stories,visual) methods, sharing details about the sample and selection process increases credibility and trustworthiness.
Use measures, indicators or metrics: We usually think about indicators as being useful for reporting and accountability to managers and donors. Indicators should also be used for providing partners, communication groups and others participating in the intervention with information about what was achieved/not achieved, and the importance of the indicators for their community. When using the data from indicators in this way, it is important to acknowledge that the information is simplified and partial, and that other types of information are usually needed to make informed decisions about the intervention.
Combine qualitative and quantitative data: A key part of being accountable is rigour. Combining data from different data collection methods boosts the rigour by providing different perspectives and ways to understand a problem.
Analyse data: Those involved in data analysis are in a powerful position of meaning making. It is important that the processes used to analyse data are rigorous, systematic and transparent.
Manage Data: C4D emphasises good data management and ownership processes that are respectful, ethical, and responsible. It is important to agree to policies and processes that prevent or minimise harm (especially for vulnerable groups). These discussions should take place before, during and after the data collection.
Understand causes (to answer questions about causes and contributions)
Investigate Causal Attribution and Contribution
A central question in RM&E from an accountability perspective is 'what has been the impact (or contribution) of C4D to observed changes'. Answering this question rigorously requires selecting carefully from three causal analysis strategies:
- Compare results to a counterfactual;
- Check the results support causal attribution;
- Investigate possible alternative explanations).
Synthesise Data from a Single Study/Evaluation: By undertaking data synthesis processes we can make findings based on different sources of evidence and voices. This is useful tool for accountability to partners and community groups, and to donors and managers.
Report and support use
Identifying Reporting Requirements: Reports from R,M&E are usually focused on satisfying the needs of donor and managers. These are important users, but it is also important to think about the reporting needs of other groups we are accountable to. This includes partners, community groups, local institutions and other stakeholders.
Developing recommendations: How can social accountability principles be used to ensure that recommendations from stakeholders are heard and meaningfully considered?
Supporting use: To achieve social accountability it is critical that recommendations from different stakeholders are heard and meaningfully considered by decision makers. Committed and transparent processes to ensure that the findings (both positive and negative) from R,M&E are used is an important part of accountability.
Challenges and strategies
Audience for reporting
Systems like Results-Based Management tend to prioritise reporting to managers and donors, rather than accountability to partners and community groups. C4D can lead the charge in advocating for a multi-dimensional understanding of accountability, as set out in the SDGs
What does success look like?
In Results-Based Management, accountability is usually judged based on how well the initiative followed planned processes. This is not as suitable in complex situations or where experimentation and adaptability are key.
The following tasks and approaches can be useful for thinking about alternatives:
- Determine what 'success' looks like: this task is about being transparent about what 'success' means to different groups. Experimentation and learning could be part of what is valued.
- There are different ways to develop M&E Frameworks, including using Outcome Mapping, which sets out different ways of monitoring performance. Alternatives are discussed in Develop Planning Documents (Evaluation Plans and MnE Frameworks).
Fear of failure
Too much focus on ‘accountability’ for ‘results’ can make people fear failure. In C4D where outcomes are less controllable and predictable, teams might start limiting the objectives they set. This can also lead to less collaboration, where sections or organisations (in a joint project) only work on areas that they are accountable/responsible for. The Theory of Change is a useful way to create a holistic overview of how we think different parts of the project contributions to change (see Develop program theory or logic model). Concepts like the sphere of control, sphere of influence, and sphere of concern, borrowed from Outcome Mapping, can be useful for identifying the degree to which teams should be expected to be accountable for changes.
A common perception within some organisations is that high quality and credible evaluations require quantitative data
There are certainly ways to ensure rigour and quality in mixed-methods and qualitative designs. The following tasks can be useful:
- Use the Questions-Led ME Framework resource, which is based on the idea that methods should be chosen based on how well they will answer questions in the local context. These steps help ensure well-considered decisions.
- Define ethical and quality evaluation standards: this task includes advice for identifying quality standards, and how to make sure these are followed.
- Identify primary intended users and Decide purpose: these interlinked steps should clarify who will use the R,M&E and what they will use it for. If the users or the purposes requires being sensitive to what certain people believe is 'credible', that might in fact be a good reason to select particular
by Jupp, D. & Ali, S. I. (with contribution from C. Barahona).
This case offers an alternative model for creating indicators. Participatory process were used to develop and measure progress against indicators of community empowerment. Accountability in this case is primarily maintained at the community level, but the data generated was repurposed for upward reporting requirements.
U-Report is an SMS-based, user-centered social monitoring tool which supports social accountability. U-Report supports social accountability by providing a platform on which citizens can voice concerns and priorities and provide feedback to governments, development agencies and other decision makers.
'C4D: Accountable' is referenced in:
- Communication for Development (C4D) :