Dealing with complexity in development evaluation - A practical approach

This book offers practical guidance to policymakers, managers, and evaluation practitioners on how to design and implement complexity-responsive evaluations that can be undertaken in the real world of time, budget, data, and political constraints.

This resource and the following information was contributed to BetterEvaluation by Estelle Raimondo, George Washington University, Washington, DC.

Authors and their affiliation

Michael Bamberger - Independent Consultant

Jos Vaessen - Maastricht University

Estelle Raimondo - George Washington University , Washington, DC

Key features of Dealing with complexity in development evaluation

Recognizing that complexity calls for innovative, conceptual, and methodological solutions, this unique book offers practical guidance to policymakers, managers, and evaluation practitioners on how to design and implement complexity-responsive evaluations that can be undertaken in the real world of time, budget, data, and political constraints. Introductory chapters present comprehensive, non-technical overviews of the most common evaluation tools and methodologies, such as theory-based evaluation, mixed-methods evaluation, impact evaluations, and reviews and syntheses. Additional content addresses more cutting-edge material, including approaches stemming from systems thinking and complexity sciences, the use of ICT in data collection and analysis, leveraging big data, and realist syntheses. The book also includes six case study chapters to illustrate examples of various evaluation contexts from around the world.

How have you used or intend on using this resource?

I have used this resource for three different purposes:

1. Teaching the basic concepts of complexity as they relate to the work of evaluators in development context.

  • The book offers a succinct review of the main literature on complexity and it proposes a straightforward framework to think about the main sources of complexity (the nature of the intervention, embeddedness in a system, institutions and stakeholders, causality and change, and the nature of the evaluation process)
  • The central message of the book is that while there is no standard recipe for dealing with complexity in evaluation, there are a number of approaches that can produce operationally useful and methodologically sound complexity-responsive evaluations. The book then goes through the main approaches in a practical way.

2. To communicate with evaluation commissioners about addressing complexity in evaluation

  • Chapter 2 proposes a tentative checklist for assessing the levels of complexity and helping managers determine which aspects of complexity should be dealt with
  • Chapter 3 of the book is dedicated to evaluation managers and commissioners and offers a step-by-step approach in the design and management of complexity responsive evaluation
  • Each chapter ends with a section on "practical applications" which speaks to non-evaluation experts and draw the key lessons from each chapter

3. To learn about more advanced approaches to deal with complexity

  • The book has an entire section on emerging data and innovative techniques to deal with complexity. The chapter on big data is particularly interesting as it not only goes through the main usage of big data in evaluation but also addresses the main challenges that big data presents for the future of evaluation 

Why would you recommend it to other people?

The book operationalizes a complexity perspective into mainstream development evaluation. It caters to a variety of audiences and balances advanced content, with practical tools and illustrative cases.

To evaluation students: I would recommend it for its review of the main evaluation methodologies and as an introduction to more advanced content on complexity

To advanced evaluators: the book covers advanced topics and proposes novel approaches for dealing with complexity, while always keeping in mind the real-world constraints in which development evaluators operate.

To evaluation managers and commissioners: the book offers real-world examples of complexity-responsive evaluations, a chapter dedicated to managerial challenges, and a chapter focusing on institutional complexities that poses particular challenges for the design, process and use of evaluation. 

Sources

Bamberger, M., & Vaessen, J., & Raimondo, E. (2015) Dealing With Complexity in Development Evaluation - A Practical Approach. SAGE publications.