Funder evaluator dialogue on complexity

This presentation by Jonny Morell focuses on the need to promote dialogue between evaluators and program funders/designers on the implications of complex behavior.

This resource and the following information was contributed by Jonny Morell.

Authors and their affiliation

Jonathan Morell, Editor, Evaluation and Program Planning

Key features

This youtube video focuses on the conversations needed between evaluators and program funders/designers. Effective social programs and useful evaluations require more attention to complex behavior than is currently the case. Examples of relevant behaviors include:

• Scaling
• Growth patterns
• Realistic timeframes
• Discontinuous change
• Unpredictable outcomes
• Unpredictable outcome chains
• Consequence of small changes
• Feedback loops among outcomes
• Inequitable distribution of benefits
• Joint optimization of uncorrelated outcomes

Evaluation methodologies are not the limiting factor in recognizing these behaviors. Almost all of them can be addressed with traditional quantitative and qualitative methodologies. The problem is program design because including complex behavior in program theory has disturbing implications program funders. The reasons for this discomfort are powerful and legitimate, and must be respected. Still, awareness about the implications of complexity for program activity and outcome needs to be increased. Increased awareness will lead to the evaluation of more realistic program theories.

How have you used or intend on using this resource?

I use the content of this video when I work with clients to shape their understanding program operation and outcome. This tactic is valuable because evaluation is too often based on program theories that are blind to complex behavior’s role in how programs work and what they accomplish. That myopia results in evaluations that do not guide program improvement, and which fail to detect or assess unintended consequences.

Why would you recommend it to other people?

Program theories are models that deliberately simplify reality. They omit some elements for the purpose of highlighting relationships among others. These simplifications are desirable and necessary. However, good evaluation requires that wise choices be made with respect to what goes into a model and what is excluded. Those choices cannot be made without purposeful consideration of complex behavior. Some complex behaviors will need to be included, and some will not. Program theory, and hence useful evaluation, suffers when these choices are not deliberated.