Causal link monitoring

Causal link monitoring (CLM) is an approach to designing and implementing monitoring, evaluation and learning (MEL) systems that prioritise information for managing adaptively in complexity. CLM helps project planners and managers decide what, when, and how to monitor and evaluate in order to manage performance adaptively.

Overview

CLM combines implementation design and MEL to support the adaptive management of projects. CLM is not a data collection method. It helps project planners and managers identify the processes required to achieve desired results and whether and how they have occurred.  This is done through continuous cycles of designing, monitoring/evaluating, and re-designing over the project’s life, which enables adjustments to improve effectiveness. This approach strengthens the data-to-design relationship, allowing data to guide adaptive implementation and ensuring data collection evolves with the project.

Result-producing processes specify the causal links between results in a logic model or theory of change (TOC)  framework—in other words, the processes between results. CLM focuses on how specific individuals or organisations use one result to achieve another result.

 

Principles for this approach

Causal Link Monitoring attends to complexity-aware principles:

• Attend to performance monitoring’s blind spots.
• Attend to interrelationships, perspectives, and boundaries.
• Synchronising monitoring with the pace of change.

Steps in the process

In CLM, planners start by creating a logic model to help document predictable, agreed-upon elements of the project. Next, they refine the causal links by describing the processes that will transform results at one point in the causal chain to the next. Planners are often less certain about these result-producing processes. Finally, the CLM logic model is enhanced with information about two important sources of uncertainty, contextual factors that may influence the project and diverse perspectives on the problem and its solution.

The CLM process can be broken down into seven steps: three for project design, three for monitoring, and a final step in which monitoring data informs redesign:

1. Build a logic model or TOC.
2. Identify assumptions about causal links.
3. Enhance the logic model with diverse perspectives and contextual factors.
4. Prioritize areas of observation.
5. Collect monitoring data.
6. Interpret and use monitoring data for adaptive management.
7. Revise the logic model.

Key characteristics/concepts of causal link monitoring

Streamlined Monitoring, Evaluation and Learning (MEL)

Using CLM’s complexity-aware theory of change, MEL planners identify data needs on a visual representation of the TOC, ensuring adequate coverage of the project’s strategic points and potential for influence from the context. They design ways to monitor and evaluate the project's uncertain, contested, emergent, and dynamic aspects and context. By integrating complexity-aware MEL and performance management, teams can make better choices about what data they will need to manage adaptively and when they will need it. This counterbalances tendencies to overburden the MEL system.

Iterative Project Design

Using CLM, the MEL plan forecasts when the team should revisit specific points in the TOC, what data will be available, and what resources are needed to support each project review.

Adaptive management

A CLM-based MEL system is explicitly designed to evolve over the life of the project. The enhanced TOC reveals when data is most relevant during implementation. More importantly, project re-design is foregrounded, making it clear that initial predictions about what data will be useful will likely evolve as the project evolves. This prompts MEL planners to specify processes and provide resources to adjust the MEL system to ensure continued alignment with project implementation.

For more information about the key characteristics/concepts of causal link monitoring, see the following blog:

Features consistent with a causal pathways perspective

A causal pathways perspective on evaluation focuses on understanding how, why, and under what conditions change happens or has happened. It is used to understand the interconnected chains of causal links which lead to a range of outcomes and impacts. These causal pathways are likely to involve multiple actors, contributing factors, events and actions, not only the activities associated with the program, project or policy being evaluated or its stated objectives. Causal Link Monitoring can be used in ways which incorporate the following features of a causal pathways perspective:

• Addressing power and inclusion: The perspectives of different stakeholders are included in when enhancing the theory of change and in data collection; they may be sought when reviewing and interpreting monitoring data and recommending adaptations to implementation, as well.
• Valuing actor’s perspectives: CLM specifies the inclusion of diverse perspectives on the situation and the intervention (e.g., participants, implementing and donor agencies, especially those who are left out or opposed to the intervention) to inform enhancements to the theory of change.
• Articulating explicit causal pathways: During program design, CLM prompts planners to articulate assumptions about how key actors will transform results at each step in a causal chain. CLM acts as a continuous “reality test” of the theory of change, helping managers detect and adapt to unpredicted causal links or contextual factors. Implementers revisit and update the design throughout the life of the project.
• Paying attention to a range of outcomes and impacts: CLM seeks information about positive and negative intended and unintended outcomes and impacts.
• Understanding contextual variation: CLM guidance on how to collect monitoring data includes direction to explore the effect of contextual factors and to specifically ask: “What contextual factors are influencing result-producing processes?”
• Using an iterative, bricolage approach to MEL design: The CLM process of MEL system design uses a bricolage approach, which carefully balances the strengths and limitations of each method. CLM MEL systems combine monitoring, evaluation, and learning methods to collect both performance data and complexity-aware data throughout the project's life. As an iterative process, each design-and-monitor cycle generates new insights into causal pathways, which are then further explored and tested in subsequent iterations.
• Drawing on a range of causal inference strategies: CLM prompts planners to enrich their development hypothesis with assumptions about how key social actors will transform results in a causal chain into new results. A single causal link may include or even require multiple processes to achieve this transformation.
• Complexity-aware perspective on causal pathways: CLM explicitly includes and examines the influence of contextual factors and diverse perspectives on the project, sensitising implementers to what cannot be predicted. CLM integrates theory-based and complexity-aware MEL. Theory-based approaches describe how the project intends to influence the context and contribute to intended outcomes, and complexity-aware approaches attend to the unpredictable ways in which the context influences the project.

Background

History of this approach

The “Causal Link Monitoring” brief (Britt et al., 2017, p 1) describes causal link monitoring as an iteration of Process Monitoring of Impacts:

CLM was included as one of the recommended approaches in the 2013 USAID Discussion Note on Complexity-Aware Monitoring. The CLM brief expanded on the original elements of the Process Monitoring of Impacts approach and the approach was renamed Causal Link Monitoring.

How does CLM differ from results-based management monitoring and evaluation frameworks?

Results-based management MEL systems are poorly suited to support adaptive management because they monitor predetermined results along change pathways predicted by funders and implementers. All too often, the theory of change, and hence the evaluations and MEL systems based on it, do not include the context, even when contextual factors are likely to influence project implementation and when project effectiveness would be enhanced by learning about and adapting to the context or seeking to change it.

CLM MEL systems differ from results-based management MEL systems in two important ways. First, rather than relying solely on results indicators, CLM monitors the causal links leading to results, providing useful data before results can be measured. Second, CLM expands the theory of change to include unpredictable aspects of an intervention and emergent results and then designs monitoring, evaluation and learning methods to track those. By integrating complexity-aware approaches with results-based monitoring, CLM tells a more complete and accurate story about the intervention in its context.

Methods that are part of this approach

BetterEvaluation defines an approach as a systematic package of methods. The Rainbow Framework organises methods in terms of more than 30 tasks involved in planning, managing and conducting an evaluation. Some of the methods used in CLM and the evaluation tasks they relate to are:

• Develop an understanding of the situation: Causal Link Monitoring includes various methods including stakeholder analysis to undertake an analysis of contextual factors such as the local economy, social norms, regulations and other projects.
• Develop a theory of change: CLM uses the method Articulating mental models to learn from the diverse views of project partners, intended beneficiaries and other stakeholders about how change occurs to inform the refinement of the theory of change.
• Collect or retrieve data about context, activities, results and other factors: The choice of methods is determined by what is most appropriate for collecting data from the actors involved in an area of observation.
• Ensure diverse perspectives are included, especially those with little voice: Enhancing the logic model involves identifying individuals who are likely to view the situation and intervention differently from project funders, planners and implementers, then prioritising collecting data on their experiences and perspectives and incorporating this into the updated implementation approach and logic model.
• Consensus decision making: Stakeholders participate in developing or enhancing the logic model so that it represents where there is consensus and where there are different perspectives on the situation and the intervention. Differing views on causal pathways should be represented in the logic model or an alternative theory of change that can be tested during implementation.

Example

The Causal Link Monitoring Brief includes a hypothetical example (based on an actual program) of a development intervention to increase the sustainable agricultural productivity of smallholder farmers as part of a larger program to increase farmers’ income in a rural area exposed to the effects of climate change. The example has been adapted to better illustrate the Causal Link Monitoring approach and how it adds value for adaptive management both within and external to the implementing agency.

The example illustrates the seven steps of the Causal Link Monitoring process. A series of diagrams illustrate how a CLM cycle results in a more comprehensive and refined logic model. The CLM process is iterative, and the newly refined logic model becomes the starting point for the next round of CLM. As explained in the brief: 

"….during each cycle, CLM monitoring is based on an increasingly refined and evidence-based logic model and provides management more realistic and up-to-date information. This facilitates decisions about whether adjustments are needed in the way activities are implemented or outputs are used. When monitoring data signals deviance from the project design, information about the results-producing processes taking place, and multiple perspectives on the project design helps identify the reasons for deviance."

Advice for choosing this approach

What types of projects and programs would this approach be appropriate for?

CLM is appropriate for monitoring complicated or complex programs (or aspects of programs) where changes in context are likely to affect results or when the processes with results chains are unclear or disputed. It is particularly useful for interventions with long impact chains (or causal pathways) where higher-level results are produced at the end of implementation or some time afterwards. 

CLM is useful for monitoring complicated situations as it can accommodate multiple interrelationships, feedback processes, emergent phenomena, sudden changes, and differing perspectives. It is especially useful when information is scarce, the context is dynamic, and consensus is elusive. In those circumstances, it’s critical to make good decisions about how to use limited MEL resources. 

What types of evaluations is CLM appropriate for?

CLM can be used for discrete evaluations or to design an MEL system as a whole or in part.

CLM is intended for use throughout the life of a project, from the early stages of design through the final assessment. Once a project is underway, there are several key points when the approach may be introduced:

• Developing a theory of change that pinpoints complexity.
• Designing an M&E plan and making decisions about what to monitor, why and how.
• Planning an evaluation to inform adaptive management.
• Conducting a mid-course project or program review with stakeholders

CLM is appropriate for both learning and accountability purposes and can inform adaptations needed during program implementation.

The blog on CLM provides more information about when it is useful.

What level and type of resources are required for this approach?

The time and money required for CLM depend on the frequency of CLM cycles, the resources required to engage with a range of perspectives, and the extent of data collection. However, being strategic about what data is needed and when it is needed can reduce the data collection burden.

How might this approach be part of an effective overall evaluation design?

CLM can be used to design monitoring, evaluation and learning systems that integrate complexity-aware MEL for complicated or complex aspects of an intervention alongside indicator-based performance monitoring for aspects of the intervention where predictability is higher (in the Cynefin framework’s “simple” or known zone). The CLM brief discusses how this approach complements other evaluation approaches and methods:

"Although it is essentially a design and monitoring approach, CLM can be valuable for evaluation. For example, it offers a good basis for theory-based evaluation approaches, such as contribution analysis and process tracing. Evaluators can build on a validated and updated logic model and use already collected monitoring data to assess the inference of the causal claims expressed in the model. With its iterative and flexible nature, CLM can be used in formative evaluations aimed at improving ongoing interventions, notably by fine-tuning the logic model and clarifying key causal links. Finally, CLM is an ideal complement to developmental evaluation, aimed at informing adaptive management through rapid feedback and supporting ongoing learning. Developmental evaluation builds on similar premises as CLM and can make effective use of the information provided by CLM."

Causal Link Monitoring’s logic model describes processes leading to intended results as well as the ways that complexity may influence the causal pathways. Users should apply appropriate data collection methods to track unplanned causal links and capture emergent outcomes using evaluation approaches such as Outcome Harvesting, Most Significant Change or QuIP, which are well-suited to identifying a broad range of results, intended or unintended and positive or negative, and seeking causal explanations for those results. Because CLM integrates complexity-aware and theory-based approaches, it helps to ensure appropriate coverage of both intended and emergent results.

 

Advice for using this approach effectively

A hypothetical example of the use of causal link monitoring is included in the Causal link monitoring brief and provides tips for each step of the process (Britt et al., 2017, p. 16-30):