Demystifying systemic thinking for evaluation in post-normal times

Anne Stephens. Shravanti Reddy. Ellen D Lewis.'s picture 13th November 2018 by Anne Stephens. Shravanti Reddy. Ellen D Lewis.

When you suggest to someone that you’re interested in using systems thinking in your project, you’ll  often  hear “we tried that once…” or “I like the idea of it but I’m not really sure how I would apply it” or “I get what it is, but I don’t really get what it is!”  These are all comments that I have heard over the last three years when asked, “So what are you working on?”

I’m a systems thinking theorist and for the last three years I’ve been working with my colleagues Ellen Lewis (Ethos of Engagement Consulting) and Shravanti Reddy (UN Women’s Independent Evaluation Service) to write a comprehensive field guidance on systemic evaluation. The Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized voices (ISE4GEMs) Guide is now on the BetterEvaluation website.

The three of us have embraced systemic thinking and wanted to provide guidance that demystifies systemic thinking and complexity science concepts, walks one through the process of evaluation using systemic thinking, and provides tools, examples and activities, to ‘be’ an inclusive systemic evaluator. 

As Bob Williams, a man who needs no introduction on this forum, said about the ISE4GEMs guide on the Pelican listserve recently:

What makes it almost unique is that it is essentially a soup to nuts evaluation manual rather than a general description of a particular evaluation approach.  

So what exactly is ‘systemic evaluation’?  In the guide, we make an important differentiation between ‘systematic’ and ‘systemic’. Put very simply, to think systematically, one engages in a process that can be thought of as, predictable, thorough and controlled, as a way to think about parts of a system that may interact. While to think systemically encourages us to be more critical about how those parts interact. What are the opportunities, constraints and relationships of parts within a system that tell us much about the system as a whole?  We write:

Systemic evaluation asks questions to capture the conditions and changes relevant to an intervention, the changes it produces and opportunities for learning and empowerment. In the course of an evaluation, one may uncover what else was going on—the external effects, spill over of other efforts or policies,  uncontrolled events (such as political conflict or environmental disasters), or unexpected facilitators or inhibitors of change that may or may not have been part of the original plan. (ISE4GEMs, p. 8)

We think that these skills are apposite for working in uncertain times, which both characterises and clichés life in the 21st Century, but more specifically, if we are to meet the goals of the 2030 Agenda: the SDGs.

Professor Thomas Schwandt summed it up perfectly in a captivating presentation he gave at the 2018 European Evaluation Society conference. He argued that we live in ‘post-normal’ times. Drawing on the seminal work by Silvio Funtowicz and Jerome Ravetz, post-normal science contrast to traditional problem-solving strategies by accounting for complexity, emergence and plurality. In a democratic mode it responds to what might be appropriate, rather than what should be done. He suggests we think about evaluation in new ways, as we work in a post-normal evaluation era. One that calls for us to use systemic thinking to make sense of new-normal levels of social, political, economic and ecological complexity.

And this is what, we believe as authors, we are endeavouring to achieve with the release of the ISE4GEMs.  More than a description of systems thinking, it provides a primer in systemic evaluation for practice. Our intention is to be your thought partner, to help you reflect on the work you are being asked to do as you unfold your understanding of the evaluation. We then provide guidance with the detailed steps one might take, to do a systemic evaluation. We created tools to work with and resources to draw on. And, further to this, we’ve used the approach, as we were writing it. So many examples from our practice and others are embedded in the guide.

Why is this different?  Because there are not too many guides that have reviewed the literature and carefully chosen techniques, strategies, ideas and concepts from across the disciplines of evaluation, systems sciences and feminism, and spill over into a range of social and environmental science disciplines as well. We have selected key ideas and established a framework that generates practitioners’ knowledge and capacity to ‘do’ systems thinking, rather than just have a better general awareness of the theory.

Get involved

We invite all evaluators, emerging or experienced, who are interested in using the ISE4GEMs to have a go and let us know as your thought partners what works, what doesn’t and how we can make the approach better. We have an active research program to investigate inclusive systemic evaluation. We want people to experience the approach and tell us how it needs to be improved, that includes the ISE4GEMs toolkit, theory (Part A) and guide (Part B). Our community of practice will shape the ISE4GEMsV2 and together we can make better sense of our complex world.

We'd love to hear your thoughts and questions in the comments below.

Read more

Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized voices (ISE4GEMs): A new approach for the SDG era

This guide by Anne Stephens, Ellen D. Lewis and Shravanti Reddy, provides an introduction to the Inclusive Systemic Evaluation for Gender equality, Environments and Marginalized voices (ISE4GEMs) approach and includes both a summary of the key theoretical concepts and guidance and tools for applying it in practice.  Read more

A special thanks to this page's contributors
Author
Evaluator, Senior Researcher & Lecturer, Director, Ethos of Engagement Consulting and Lecturer, James Cook University.
Cairns, Australia.
Author
Evaluation Specialist, UN Women.
New York, United States of America.
Author
Global Organization Development , Ethos of Engagement.
Kingston upon Hull, United Kingdom.

Comments

Anonymous's picture
William Liley

Thanks for doing all this work. I think there is great potential in the process and tools and strategies you have drawn together. Capacity to work with communities and deliberately seek to learn about the whole range of effects at local levels is the piece that has been missing in evaluation practice. I think the potential for this approach might also include the highly complex environments within the developed as well and for programs in emerging economies. Takes a while to look through , especially for novel applications or new practitioners and highly recommended, looking forward to the primer/this is how we used it series... keep up the great work!!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.