Institutionalizing evaluation: A review of international experience

This report, written by Bertha Briceño and Marie M. Gaarder for the International Initiative for Impact Evaluation (3ie), compares experiences of institutionalizing government evaluation efforts to consider the lessons learnt for countries that may want to start down that road.

Using the case studies of Mexico, Colombia, Chile, South Africa, and China, demonstrates the different contexts and approaches that are available to ensure mandatory impact evaluation of social programs takes place. 

Excerpt

"A successful institutionalised Monitoring and Evaluation (M&E) system may look different in different contexts and cultural environments, nevertheless there are the same trade-offs and considerations to be made. With the main objective of monitoring and evaluating the performance of governmental programs, the oversight body should enjoy a high degree of independence, which ultimately translates into higher external credibility, as is the case of Mexico’s CONEVAL. Legitimacy of the evaluation effort can also be attained through the establishment of competitive and open procurement processes for the contracting of external evaluations, as is the case in Colombia. The risk to credibility when the system is located under the executive, such as in Chile, can be mitigated with other provisions such as a commitment to public disclosure.

The gains from being ‘outside’ of government may come at a cost: as the system becomes separated from internal budget or planning authorities, it may have less power to enforce or exert direct influence over the objects of oversight. In the case of Chile, the evaluation unit is located within the Budget Division of the Ministry of Finance and a formal legal mandate requires evaluation of public programs, lending it strong enforcement capabilities. The risk of low enforcement capabilities can be addressed in alternative ways, however. Support from Congress, fluid communication and promotion of alliances with central authorities, are common strategies to mitigate weak enforcement of recommendations. A complementary strategy to enforce adoption of recommendations is generating a tradition of utilization as a managerial tool rather than a control tool; this is, generating ownership of evaluation by the program’s management."

Contents

• Country cases: from federal evaluation body to the learning authoritarian state  8
  • The case of Mexico: the National Council for the Evaluation of Social Policy, CONEVAL 8
  • The case of Colombia: the National System for Evaluation and Management for Results, SINERGIA  13
  • The case of Chile: the Management Control Division at DIPRES  17
  • The Case of South Africa: the Government-Wide Monitoring and Evaluation System  22
  • The case of China: the state of current evaluation efforts  23
  • Other Interesting Institutionalisation Experiences 24
• The structural design of an M&E system: balancing trade-offs  28  
• Measures of success 30