Evaluations made in Portuguese: the Lusophone space
This guest blog by Elsa de Morais Sarmento, Carla Félix, and Mariana Branco discusses the importance of language to credible evaluation in Lusophone countries.
Portuguese is the sixth most natively spoken language in the world and Portuguese-speaking countries are home to 267 million people located in four continents with a common language, a shared history, and cultural similarities. The Community of Portuguese Language Countries (Comunidade dos Países de Língua Portuguesa or CPLP) gathers nine countries: Angola, Brazil, Cabo Verde, Equatorial Guinea, Guinea-Bissau, Mozambique, Portugal, São Tomé and Príncipe, and Timor-Leste.
In all Lusophone countries, there are exceptional opportunities to strengthen evaluation activities, with governments playing a more active role in demanding and managing evaluations and using evidence. However, these countries must explore these opportunities within their considerably different political economy settings. Donors and partners can provide opportunities for learning through country-led support.
In order to discuss this, a group of Lusophone M&E champions gathered at a roundtable to discuss “The road ahead for evaluation in the Lusophone world: resilience and action in critical times” during the 2018´s European Evaluation Society (EES) biannual conference in Thessaloniki. A summer of the findings is available here.
Language as the first barrier to credible evaluation in Lusophone Countries
Along with the poor knowledge of the local context, language is currently the main barrier to building, disseminating and using credible and contextualized evaluation in Lusophone countries. A basic pre-condition for Evaluations in Portuguese is that M&E practitioners master the official language and build their practice on countries’ context and self-determination culture for greater relevance and validity of findings.
Very often, the mastery of the Portuguese language is considered secondary (e.g. clearly in most language requirements in evaluation Terms of Reference). This means that donors need to set stricter language requirements in their ToRs, along with tighter guidance for true community-based research.
In Lusophone Africa, understanding the nuances of language, deconstructing historical and cultural issues, such as the traumas of conflict is a prerequisite for conducting sound research and obtaining reliable and accurate findings.
Capitalizing on opportunities for more Evaluations in Portuguese
Evaluation demand in Lusophone countries has been largely inconsistent, due to a deeply rooted historical culture of patronage and fluid policies. Moreover, few countries have adopted all the elements of a Results-Based Management (RBM) cycle and performance-based budgeting, to align expenditure with strategic goals and priorities. Underlying issues constraining the advancement of M&E efforts, with varying degrees in different countries, have to do with weak technical capacity, a reactive institutional culture, the scarcity of resources, and the lack of political commitment.
In Lusophone Africa, the demand for evaluation is still largely driven and conditioned by development partners, who still favor the comforts of their own “language” and practice. High-quality evaluations do not abound and are commissioned and managed more often by donors than by governments (with some exceptions for academic institutions). Consequently, with governments shying away from a more direct participation, evaluations are less likely to be used in policy.
There are no ready-made fast-tracking solutions for achieving M&E goals and thus supporting the global development agenda. For most Lusophone countries, sound M&E systems and capacity development are urgently required. However, it is fundamental to conduct a participatory M&E diagnostic beforehand and involve local stakeholders in this exercise. Such a needs assessment, which would be pioneered in the African context, is a steppingstone for understanding which kind of capacity and resources ought to be put in place, who are the champions, and what are the next steps. There are three important tiers to uncover the type of support needed: the individual level (such as capacity building needs), the creation of institutional systems and organizations, and the enabling environment.
However, given existing constraints and the risks of these efforts´ crowding out, they must work collaboratively to create the necessary partnerships to make this work. By drawing on current initiatives to monitor the 2030 Agenda, and by sharing evaluation experiences with other countries, regional and linguistic blocks, Lusophone countries can create and affirm their own evaluation networks. They can also contribute to the development of local evaluation capacities and sustain an endogenous demand for evaluation, ensuring that policymakers are aware of the value of evaluation knowledge to improve policymaking, and ultimately development results.
In the 2030 Agenda, there are just too many indicators for most countries to monitor. Countries need a harmonized approach for monitoring and reporting on these developments, focusing on what is relevant, feasible and possible. In addition to the attention given to governance frameworks at the national and sub regional levels (which includes budgets, human resources, and institutional capacities), effective M&E frameworks are essential, along with statistics and disaggregated data, which make up key elements of any M&E system. In this vein, national statistical systems need to be strengthened for the effective M&E of the implementation of the 2030 Agenda.
Fast-tracking towards the achievement of the Sustainable Development Goals (SDG), implies capitalizing on existing efforts to monitor the 2030 Agenda but also the African Union’s (AU) Agenda 2063 for African countries, to obtain better data to implement evidence-based policies, and adopting Results Based Management (RBM) practises through an enhanced focus on results in government and line Ministries to foster accountability and reporting.
There are ongoing efforts towards the monitoring of the 2030 Agenda that the Lusophone countries can capitalize from. This Agenda calls for follow-up and review of the national, regional and global levels to assess the progress towards the SDGs and its targets. The UN´s report on the follow-up and review of the 2030 Agenda (A/70/684) emphasized that “the United Nations system (…) stands ready to provide coherent support to the conduct of national reviews, including for strengthening the capacity of national statistical offices, data systems and evaluations.” In response to this mandate, the UNDP developed a set of guidelines to support reporting on the SDGs at the country level. These provide guidance for defining national SDG indicators, baselines for M&E and criteria for progress assessment, thus fostering inclusive and more harmonized approaches to national SDG reviews.
In the case of Africa, the United Nations Economic Commission for Africa (UNECA) developed an Integrated Planning and Reporting Toolkit (IPRT), which is a web application developed is response to the needs of African countries to adopt and integrate both the 2030 Agenda for Sustainable Development and Agenda 2063 into their national development policies, towards a more harmonized country reporting on SDGs (the demo of the UNECA Integrated planning and reporting toolkit is available here).
The extremely interesting initiative “SDG Implementation Toolkit” (more information on the SDG toolkit here) is currently being implemented by the Commonwealth (of which Mozambique is a member, and Angola has requested membership). National country strategies, objectives, indicators, baselines and targets can be inserted in this system, managed and linked to SDGs or any other monitoring framework. It allows a harmonized reporting of country strategies and direct linkages to the most important reporting frameworks on global goals. It has the potential (and willingness from the developers) to be scaled up beyond the Commonwealth group of countries, so as to be used by any country that is interested in adopting it. There are recognized synergies between the Commonwealth’s toolkit, that of United Nations Economic Commission for Africa and the African Union. Partners agreed to harmonize their suites of tools to the greatest extent for the benefit of their member states.
The Small Island Developing States (SIDS) Accelerated Modalities of Action (SAMOA Pathway) was adopted at the Third International Conference on Small Island Developing States in Samoa in 2014 to address priority areas for SIDS. This initiative includes three Lusophone countries, Cabo Verde, São Tomé e Principe and Timor-Leste. A UN Toolkit for Monitoring and Reporting on the SAMOA Pathway is currently under development for a harmonized monitoring and reporting for SIDS, in alignment with the monitoring and reporting frameworks of international agreements including the SDGs, the Sendai Framework on Disaster Risk Reduction, the Addis Ababa Action Agenda and the Paris Agreement.
Taking this agenda forward
Misreading Lusophone countries’ linguistic, historical and social nuances in evaluation work hinders the quality, rigor and usefulness of evaluative evidence. In Africa, Lusophone countries seldom have a say in the development of new or context-adapted evaluation methodologies and approaches. Better Evaluation has already made available some resources in Portuguese, namely its “Rainbow Framework tool”, called “Ferramenta de planejamento ‘Quadro Arco-íris’.
The lack of Lusophone evaluators is preventing these countries from contributing to the development of evaluation theory and practice through their own endogenous knowledge and skills.
Addressing the 21st century´s big challenges requires that all take part in devising solutions. Empowering Lusophone M&E experts is critical to ensure diversity and promote more innovative and respectful global evaluative thinking. Lusophone countries ought to take the lead in this process and map their own needs, building on their specific contexts and internal resources, but also on international initiatives, so as to strengthen their own national evaluation systems and capacities. Sound M&E systems and evaluation capacity development are urgently required.