Causal link monitoring

Causal Link Monitoring (CLM) is an evaluation approach that combines implementation design and monitoring to support adaptive management of projects, helping project planners and managers to identify processes needed to achieve desired outcomes.

Overview

Causal Link Monitoring (CLM) combines implementation design and monitoring to support adaptive management of projects. It helps project planners and managers identify the processes needed to achieve desired outcomes and whether and how these processes have occurred. This is done through continuous cycles of designing, monitoring/evaluating, and re-designing over the project’s life, which enables adjustments to improve effectiveness. This approach strengthens the data-to-design relationship, allowing data to guide adaptive implementation and ensuring data collection evolves with the project.

Result-producing processes specify the causal links between results in a logic model or results framework—in other words, the processes between results. CLM focuses on how specific individuals or organisations use one result to achieve another result

 

Principles for this approach

Causal Link Monitoring attends to complexity-aware principles:

• Attend to performance monitoring’s blind spots.
• Attend to interrelationships, perspectives, and boundaries.
• Synchronizing monitoring with the pace of change.

Steps in the process

In CLM, planners start by creating a logic model to help document predictable, agreed-upon elements of the project. Next, they refine the causal links by describing the processes that will transform results at one point in the causal chain to the next. Planners are often less certain about these result-producing processes. Finally, the CLM logic model is enhanced with information about two important sources of uncertainty, contextual factors that may influence the project and diverse perspectives on the problem and its solution.

The process can be broken down into seven steps, three for project design, three for monitoring, and a final step in which monitoring data informs redesign:

1. Build a logic model.
2. Identify assumptions about causal links.
3. Enhance the logic model with diverse perspectives and contextual factors.
4. Prioritize areas of observation.
5. Collect monitoring data.
6. Interpret and use monitoring data for adaptive management.
7. Revise the logic model.

Key characteristics/concepts of causal link monitoring

Streamlined Monitoring, Evaluation and Learning (MEL)

Using CLM’s complexity-aware theory of change, MEL planners design ways to monitor and evaluate the uncertain, contested, emergent and dynamic aspects of the project and context. Planners locate data needs on a visual representation of the TOC which helps to ensure adequate coverage at all strategic points of the project and context. This counterbalances tendencies to overburden the MEL system. Teams can make better choices about what data they will need to manage adaptively and when they will need it.

Iterative Project Design

Using CLM, the MEL plan forecasts when the team should revisit specific points in the TOC, what data will be available, and what resources are needed to support each project review.

Adaptive management

A CLM-based MEL system is explicitly designed to evolve over the life of the project. The enhanced TOC reveals when data is most relevant during implementation. More importantly, project re-design is foregrounded making it clear that initial predictions about what data will be useful are likely to evolve as the project evolves. This prompts MEL planners to specify processes and provide resources to adjust the MEL system to ensure continued alignment with project implementation.

For more information about the key characteristics/concepts of causal link monitoring see the following blog:

Features consistent with a causal pathways perspective

A causal pathways perspective on evaluation focuses on understanding how, why, and under what conditions change happens or has happened. It is used to understand the interconnected chains of causal links which lead to a range of outcomes and impacts. These causal pathways are likely to involve multiple actors, contributing factors, events and actions, not only the activities associated with the program, project or policy being evaluated or its stated objectives. Causal Link Monitoring can be used in ways which incorporate the following features of a causal pathways perspective:

• Valuing actors’ narratives: CLM draws on diverse perspectives (e.g., participants, implementing and donor agencies) to inform enhancements to the theory of change.
• Addressing power and inclusion: The perspectives of different stakeholders may be sought when reviewing and interpreting monitoring data and recommending adaptations to implementation.
• Articulating explicit causal pathways: During program design, CLM prompts planners to articulate assumptions about how key actors will transform results at each step in a causal chain. CLM acts as a continuous “reality test” of the theory of change, helping managers detect and adapt to unpredicted causal links or contextual factors. Implementers revisit and update the design throughout the project's life.
• Paying attention to a range of outcomes and impacts: CLM seeks information about outcomes and impacts, positive or negative. Although CLM has mostly focused on intended results, it can also be used to seek information on unintended results.
• Understanding contextual variation: CLM guidance on how to collect monitoring data includes direction to explore the effect of contextual factors and to specifically ask, “What contextual factors are influencing result-producing processes?”
• Using an iterative, bricolage approach to evaluation design: CLM is an iterative process. Each iteration generates new information about causal pathways to be further explored and tested in a subsequent iteration.
• Drawing on a range of causal inference strategies: CLM prompts planners to enrich their development hypothesis with assumptions about how key social actors will transform results in a causal chain into new results. A single causal link may include or even require multiple processes to achieve this transformation.
• Taking a complexity-appropriate approach to evaluation quality and rigour: CLM encourages the triangulation of performance monitoring and CLM data and actively seeks differing perspectives in the interpretation of data.

Background

History of this approach

The “Causal Link Monitoring” brief (Britt et al., 2017, p 1) describes causal link monitoring as an iteration of Process Monitoring of Impacts:

"… Process Monitoring of Impacts addressed monitoring challenges associated with multiple objectives, a broad range of implementing agents, and a large number of projects associated with Structural Fund programs. It was inspired by Outcome Mapping, especially the focus on intended behavioral change and their performance and contribution toward expected results. CLM was initially described as Process Monitoring of Impacts in Systems Concepts in Action (Williams, B., and R. Hummelbrunner, 2011)."

CLM was included as one of the recommended approaches in the USAID Discussion Note on Complexity-Aware Monitoring. The CLM brief expanded on the original elements of the Process Monitoring of Impacts approach and the approach was renamed Causal Link Monitoring.

How does CLM differ from results-based management monitoring and evaluation frameworks?

CLM differs from M&E frameworks based on results-based management which monitor predetermined results along predicted change pathways. These pathways do not support adaptive management. CLM was developed to accommodate complexity and inform adaptive management. Rather than monitoring results indicators CLM monitors the processes leading to results.

Methods that are part of this approach

BetterEvaluation defines an approach as a systematic package of methods. The Rainbow Framework organises methods in terms of more than 30 tasks involved in planning, managing and conducting an evaluation. Some of the methods used in CLM and the evaluation tasks they relate to are:

• Develop an understanding of the situation: Causal Link Monitoring includes various methods including stakeholder analysis to undertake an analysis of contextual factors such as the local economy, social norms, regulations and other projects
• Develop a theory of change: CLM uses the method Articulating mental models to learn from the diverse views of project partners, beneficiaries and other stakeholders about how change occurs to inform the refinement of the theory of change
• Collect or retrieve data about context, activities, results and other factors: The choice of methods is determined by what is most appropriate for collecting data from the actors involved in an area of observation.
• Ensure diverse perspectives are included, especially those with little voice: Involving stakeholders in a review of the logic model facilitates discussion and encourages consensus. Where consensus cannot be reached differing views on causal links should either be represented in the logic model or in an alternative theory of change that can be tested during implementation
• Consensus decision making: A decision-making method that involves reaching agreement between all members of a group with regards to a certain issue.

Example

The Guide includes a hypothetical example (based on an actual program) of a development intervention to increase the sustainable agricultural productivity of smallholder farmers as part of a larger program to increase farmers’ income in a rural area exposed to the effects of climate change. The example has been adapted to better illustrate the Causal Link Monitoring approach and how it adds value for adaptive management both within and external to the implementing agency.

The example illustrates the use of the seven steps of the Causal Link Monitoring process. A series of diagrams illustrate how a CLM cycle results in a more comprehensive and refined logic model. The CLM process is iterative, the new refined logic model becomes the starting point for the next round of CLM. As explained in the brief:

"….during each cycle, CLM monitoring is based on an increasingly refined and evidence-based logic model and provides management more realistic and up-to-date information. This facilitates decisions about whether adjustments are needed in the way activities are implemented or outputs are used. When monitoring data signals deviance from the project design, information about the results-producing processes taking place, and multiple perspectives on the project design helps identify the reasons for deviance." (Britt et al., 2017)

Advice for choosing this approach

What types of projects and programs would this approach be appropriate for?

CLM is appropriate for monitoring complicated or complex programs (or aspects of programs) where changes in context are likely to affect results or when the processes with results chains are unclear or disputed. It is particularly useful for interventions with long-term impact chains where higher-level results are produced at the end or after implementation.

CLM is useful for monitoring complicated and complex situations as it can accommodate multiple interrelationships, feedback processes, emergent phenomena, sudden changes, and differing perspectives.

What types of evaluations is CLM appropriate for?

The blog on CLM describes when it is useful:

CLM is intended to be used throughout the life of a project, from the early stages of design through the final assessment. Once a project is underway, there are several key points when the approach may be introduced when:

• Developing a theory of change that pinpoints complexity.
• Designing an M&E plan and making decisions about what to monitor, why and how.
• Planning an evaluation to inform adaptive management.
• Conducting a mid-course project or program review with stakeholders

CLM is appropriate for both learning and accountability purposes and can inform adaptations needed during program implementation.

What level and type of resources are required for this approach?

The time and money required for CLM depend on the frequency of CLM cycles, the resources required to engage with a range of perspectives, and the extent of additional data collection that may be required. However, being strategic about what data is needed and when it is needed can reduce the data collection burden of results-based monitoring.

How might this approach be part of an effective overall evaluation design?

CLM can be used specifically for monitoring complicated or complex aspects of an intervention alongside indicator-based performance monitoring for simpler elements. The CLM brief discusses how this approach complements other evaluation approaches and methods:

"Although it is essentially a design and monitoring approach, CLM can be valuable for evaluation. For example, it offers a good basis for theory-based evaluation approaches, such as contribution analysis and process tracing. Evaluators can build on a validated and updated logic model and use already collected monitoring data to assess the inference of the causal claims expressed in the model. With its iterative and flexible nature, CLM can be used in formative evaluations aimed at improving ongoing interventions, notably by fine-tuning the logic model and clarifying key causal links. Finally, CLM is an ideal complement to developmental evaluation, aimed at informing adaptive management through rapid feedback and supporting ongoing learning. Developmental evaluation builds on similar premises as CLM and can make effective use of the information provided by CLM." (Britt et al., 2017, p 13-14)

Causal Link Monitoring focuses on processes leading to intended results. While unplanned causal links may be identified, other evaluation approaches, such as Outcome Harvesting, Most Significant Change, or QuIP, are better options for identifying all results, intended or unintended and positive or negative, and seeking causal explanations for the results.

Advice for using this approach effectively

A hypothetical example of the use of causal link monitoring in the Causal link monitoring brief provides tips for each step of the process (Britt et al., 2017):