Funder Evaluator Dialogue on Complexity

This presentation by Jonny Morell focuses on the need to promote dialogue between evaluators and program funders/designers on the implications of complex behavior.

This resource and the following information was contributed by Jonny Morell.

Authors and their affiliation

  • Jonathan Morell, Editor, Evaluation and Program Planning

Year of publication

2017

Type of resource

Overview

Key features

This youtube video focuses on the conversations needed between evaluators and program funders/designers. Effective social programs and useful evaluations require more attention to complex behavior than is currently the case. Examples of relevant behaviors include:

  • Scaling
  • Growth patterns
  • Realistic timeframes
  • Discontinuous change
  • Unpredictable outcomes
  • Unpredictable outcome chains
  • Consequence of small changes
  • Feedback loops among outcomes
  • Inequitable distribution of benefits
  • Joint optimization of uncorrelated outcomes

Evaluation methodologies are not the limiting factor in recognizing these behaviors. Almost all of them can be addressed with traditional quantitative and qualitative methodologies. The problem is program design because including complex behavior in program theory has disturbing implications program funders. The reasons for this discomfort are powerful and legitimate, and must be respected. Still, awareness about the implications of complexity for program activity and outcome needs to be increased. Increased awareness will lead to the evaluation of more realistic program theories.

Who is this resource useful for?

  • Advocates for evaluation
  • Commissioners/managers of evaluation
  • Evaluators

How have you used or intend on using this resource?

I use the content of this video when I work with clients to shape their understanding program operation and outcome. This tactic is valuable because evaluation is too often based on program theories that are blind to complex behavior’s role in how programs work and what they accomplish. That myopia results in evaluations that do not guide program improvement, and which fail to detect or assess unintended consequences.

Why would you recommend it to other people?

Program theories are models that deliberately simplify reality. They omit some elements for the purpose of highlighting relationships among others. These simplifications are desirable and necessary. However, good evaluation requires that wise choices be made with respect to what goes into a model and what is excluded. Those choices cannot be made without purposeful consideration of complex behavior. Some complex behaviors will need to be included, and some will not. Program theory, and hence useful evaluation, suffers when these choices are not deliberated. 

Source

Morell, J. (2017).  Funder Evaluator Dialogue on Complexity. [Video]. Retrieved from: https://www.youtube.com/watch?v=lqynQPS8ZN4

3
Average: 3 (1 vote)
Rate this Resource:
This resource is useful for:
A special thanks to this page's contributors
Resource Suggested By
Director of Evaluation, Fulcrum Corporation and Editor of Evaluation and Program Planning.
Ann Arbor, United States of America.

Comments

Miek van Gaalen's picture
Anna Maria (Miek) van Gaalen
Rating: 
0

I think the contents of this presentation is highly relevant and extremely important, however I am wondering if this could be presented in a more user friendly way. It is a long sit, and unfortunately the sound is not the best... I would love to see an abbreviated version or a more visual shorter representation. It is highly relevant for an evaluation I am currently involved in however, due to the format, I am afraid I cannot really use it (yet). But great to know this kind of presentations are emerging! Thanks for posting!

 

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.