Foresight evaluation

Foresight evaluation is a broad concept that includes monitoring and evaluation (M&E) of specific foresight activities and using foresight as a set of tools and skills to enhance M&E practice.

On this page:

• Introduction
• What is foresight?
• Purpose of foresight evaluation
• Key foresight concepts
• Key foresight models and methods
• Tackling foresight evaluation challenges
• Useful evaluation concepts and tools
• Resources

Introduction

Foresight activities may involve creating and using possible future scenarios, identifying and tracking signals and forces of change in the environment, and educating individuals, communities, and organisations in both the public and private sectors on how to apply foresight techniques.

While using foresight techniques is not new, it is becoming increasingly important due to the growing awareness of the complexity and interconnectedness of global challenges. Integrating foresight into strategic planning and policymaking helps anticipate and navigate these complexities, leading to more effective outcomes.

An understanding of foresight practices is also increasingly important for evaluators. Evaluators are often called upon to think creatively and strategically about the future. This involves creating flexible, forward-thinking evaluation plans for interventions that can adapt to change. Such an approach shifts the focus from retrospective analysis to enhancing a program’s resilience in the face of significant change (Carden, 2023). Evaluators can no longer assume projects will remain static; they must adopt a realistic perspective that acknowledges the often complex and chaotic nature of current realities (Schwandt, 2019).

What is foresight?

“Foresight is the capacity to think systematically about the future to inform decision-making today.”

Foresight focuses on investigating the drivers of change and exploring possible futures to inform planning and policymaking. In this context, the concept of 'future' typically involves a timeframe of longer than five years, which is the timeframe typically used in strategic planning. It is important to note that foresight does not seek to predict the future since none of the futures that come to pass are exactly as imagined. However, it does support better preparation for any future that may arise and spur imagination and collective creativity.

Foresight practitioners work in all types of organisations, including government, businesses, education, communities, and foundations. Organisations may embed foresight in their strategic planning unit or have a separate department focusing on the organisation's long-term impact.

Foresight practitioners think about and use the future to better understand and navigate a world that is Volatile, Uncertain, Complex and Ambiguous (VUCA). Foresight is considered a type of systems thinking since it examines the relationships between drivers, trends, events, and signals of change.   In practice, foresight involves a range of activities, including developing alternative scenarios, environmental (or horizon) scanning, foresight training and workshops, forecasting, and identification and analysis of trends and emerging signals.

The Association of Professional Futurists' six-practices Foresight Competency model (Hines et al., 2017) provides a more detailed characterisation of foresight practitioner expertise and possible assessment areas:

• Framing: Scoping the project, defining the focal issue and current conditions. Locating the project in its context and understanding its history. Understanding the client and/or audience assumptions, knowledge, and interests;
• Scanning: Exploring and gathering signals of change or indicators of the futures that affect the project or topic in a systematic fashion. Assessing these signals for relevance and impact.
• Futuring: Identifying a baseline and generating alternative futures using current trends, issues, wild-cards, uncertainties, and events, allowing participants to see the future with fresh eyes and suspend assumptions. Using these scenarios to inform strategy and future-proof planning.
• Visioning: Developing and committing to a preferred future that is informed by the past, present, and alternative futures.  The vision, a preferred future, drives development of achievable goals and action steps.
• Designing: Developing prototypes, offerings or artifacts to achieve the vision and goals,  such as simulations, gaming, etc.
• Adapting: Enabling organizations to generate options to alternative futures that informs strategic planning, decision-making and action.

Purpose of foresight evaluation

The purpose of foresight evaluation often depends on whether ‘foresight’ is the evaluand, such as a workshop to create alternative futures, or whether it is a capacity and set of tools for expanding evaluation practice.

When a foresight activity or program is the subject of the evaluation, the purpose can vary greatly:

• In government, the focus of foresight evaluation has been primarily on accountability and whether foresight was efficiently conducted. Increasingly, there is a desire to assess whether foresight activities have influenced decision-making, as well as public and policymaker awareness.
• In foresight education, the focus is typically on assessing student learning outcomes and competency with new measurement tools that assess changes in areas such as students’ futures literacy, readiness, and consciousness.
• Corporations and nonprofits are often interested in the quality of the foresight process and whether it achieved its desired impacts, including informing strategy and future-proofing actions.
• Communities may engage in participatory foresight to co-create a preferred future. Methods to assess the strengthening of civil society, influencing decision-making, and developing creative and visionary images of the future are useful here.
• More generally, the purpose of foresight evaluation is broadening to include learning that contributes to improved field credibility and greater capacity to navigate uncertainty and change.

When used in evaluation contexts more broadly, foresight is a construct and set of methods that frees evaluators up from assumptions and views based on the past as well as a linear and overly optimistic understanding of the future. Foresight methods “bring the future into the present” and enable evaluators to engage participants in future-proofing a program from design to implementation (Carden, 2023). For example, Jess Dart and Emily Gates (2024) used three foresight methods—identifying trends and weak signals, alternative scenarios, and the Three Horizons model—as part of theory of change process for the Difference Incubator program.

Key foresight concepts

Before considering the evaluation of foresight activities, some core concepts related to foresight are useful to understand.

Change

Understanding, anticipating, and navigating different types of change is a core feature of foresight work. In the current era, change can be described as Volatile, Uncertain, Complex, and Ambiguous (VUCA). Foresight practitioners characterise change, particularly disruptive events, future developments, risks, and opportunities, through systemic methods, such as horizon scanning, scenario development, the futures wheel, trend analysis, and forecasting (see below).

Futures plural

Rather than assuming a specific event, outcome, or future, foresight thinks about and prepares for many eventualities and multiple possible futures that are well beyond the usual time horizon of a program evaluation. Seeing the world through multiple lenses helps to make choices that consider changing conditions and manage complexity and ambiguity.

Futures literacy and anticipation

Futures literacy is a skill that allows people to better understand the role that the future plays in what they see and do. People can be trained in 'using-the-future', strengthening their ability to anticipate things they might not otherwise (Miller, 2022). Exploring possible futures will likely uncover elements that reduce the risk of complete surprise. This element of anticipation reduces the likelihood of inadequate preparation or late responses to important aspects which could have been anticipated.

Key foresight models and methods

Some of the most common foresight methods and models include: 

• Backcasting: Determining the steps or actions that need to be taken to achieve a preferred future.
• Causal Layer Analysis (CLA): Process to identify the driving forces and world views underpinning diverse perspectives about the future. Can be used to produce a shared view of possible futures, organizational strategy.
• Delphi method: Gathering opinions from subject experts on the future and prioritizing issues of importance;
• Forecasting: Estimating what will happen by focusing on past and current information to predict future situations. It assumes that past trends will continue to be solid indicators of what is most likely to happen in the future.
• Futures Cone: A model that brings together probable, plausible, possible, and possible futures.
• Future Triangle: A simple method to map three competing factors: the pull of the future, the push of the present, and the weight of history. The analysis the interaction of the three dimensions can be used to develop a plausible future.
• Futures Wheel: Visually explore and understand the direct and indirect impacts and implications of a driver, issue or trend.
• Horizon Scanning: Looking ahead by systematically tracking the early signs of change in a specific policy, program, or strategy environment. Exploratory.
• Scenarios: Stories about the alternative ways the external environment might develop in the future.
• Trend Analysis: Tracking key factors in the environment that are changing and have directionality, Are multiple models to organize trends, e.g., STEEPV, PESTLE.
• Three Horizons model: A model for showing how strategic issues change over time that links foresight to planning and policy. Horizon 1 is the present system. Horizon 2 or the medium term is when early signs of influential factors and trends identified in Horizon 1 become important. It is a transition space.  Horizon 3 is a new system that is minimally present in the first Horizon and could be a space of transformation.
• Wind Tunneling: Understand which strategies are most resilient, and what to prepare for in the event of a particular scenario.

(Source: Government Office for Science, 2017; Save the Children & School of International Futures, 2019; CoLab, n.d.; UNDP Global Centre for Public Service Excellence, 2018)

Tackling foresight evaluation challenges

Evaluating activities that consider the future or a set of futures that have not happened yet or teaches people how to develop a futures mindset distinguishes foresight evaluation from other types of evaluation, where a program or policy has been implemented and outcomes can be identified. Evaluating foresight requires creativity in design, methods and knowledge of compatible areas in evaluation, such as developmental evaluation, systems thinking, and complex adaptive systems.

Common challenges in foresight evaluation identified by the Association of Professional Futurists Foresight Evaluation Task Force (2022) and Gardner et al. (2024) are listed below.

Certainty vs uncertainty

Commissioners and users of foresight work typically want to know whether foresight will reduce uncertainty in decision-making and strategic planning or whether a foresight process worked as intended, e.g., identifying signals of change that are likely to happen. Foresight acknowledges and explores uncertainty, educating project participants and decision-makers and helping them deal with it better, such as identifying wild cards (low probability, high-impact events). Evaluators can surface assumptions about certainty, identify relevant questions for the evaluation, and manage stakeholder expectations.

Participant concerns with uncertainty can be addressed with approaches that support participant engagement and co-ownership of the evaluation process. For example, empowerment evaluation is a participatory approach designed to provide groups with the tools and knowledge to monitor and evaluate their own performance and accomplish their goals. It is also used to help groups accomplish their goals (Fetterman et al., 2018).

Complexity

Evaluators contend with complexity at two levels in foresight evaluation. First, the future itself is complex and has components—time, change, values, culture--that interact in ways that may be inexplicable. Second, foresight activities are themselves complicated. They may engage many stakeholders with diverse perspectives and areas of expertise, develop or work with forecasts in multiple arenas—social, technology, environment, economy, and policy—and create detailed, complex scenarios. It is important to understand the features of complexity and be comfortable with nonlinearity. The wicked problems framework or characterising intractable problems that do not lend themselves to easy resolution allows evaluators and stakeholders to view an initiative in a fundamentally different way. For example, in a public sector foresight context, the wicked problems framework can be a helpful tool to understand the underlying complexity of policy issues and the difficulty in achieving consensus on a particular solution, such as global warming (Sherman & Peterson, 2009).

Some foresight methods, such as alternative scenarios and the future wheel, are also useful here, as they are designed to characterise and manage complexity. Other useful approaches include complex adaptive systems models, which can provide frameworks that can inform evaluation design, and evaluation methods and approaches, such as contribution analysis, contextual analyses (e.g. realist evaluation), and social network analysis. These can be used to help navigate complexity and to ‘connect the attribution dots’ and provide a more robust understanding of the role of funding, key foresight tactics, and how a policy change came about, or possibly important lessons learned if a change did not occur (Sibthorpe et al., 2004). More information can be found on the complexity thematic page.

Long time horizon and determining impacts

A particular challenge in evaluating foresight is the difficulty in identifying the changes due to a foresight activity because of the long time it takes for foresight work to manifest its impacts, for example, when foresight is being used to inform policy design. Like other programs and policies with a long-time horizon, evaluators must deal with stakeholders' expectations around what should be measured. In particular, stakeholders need to understand that foresight cannot predict the future, and there are other more salient outcomes, such as informing and influencing decision-making.

Focusing on process measures and the implementation and quality of foresight work, which can be measured immediately, are valuable in their own right, particularly to foresight practitioners and program participants who want to know how well the initiative is proceeding and areas for improvement.

Considerable work has been done to develop foresight impact schemas and typologies. Johnston (2012) provides a comprehensive review of eight frameworks to evaluate policy outcomes and the impacts of foresight. It is a useful repository for identifying impacts in diverse policy settings, such as matching impacts to six functions of foresight for policymaking and sorting impacts by type (accountability, justification, and learning).

Additionally, the impact of foresight can be ensuring that something does not happen (i.e., an outcome that is avoided), such as when foresight is used for preparedness and prevention. Evaluation of counterfactuals and what didn't happen and unintended consequences are meaningful information in the context of a foresight evaluation.

Attribution

Like other complex and uncertain evaluands, such as advocacy initiatives, the causal chain between a foresight intervention and impact can be unclear and long. If determining attribution is a priority, theories of change and approaches such as Most Significant Change and systems thinking methods can make the linkages between initiative elements more transparent. In settings with multiple contributing factors where foresight is just one component, such as influencing decision-maker support, evaluators need to educate stakeholders on the value of focusing on contribution rather than causation. Tools like contribution analysis, designed to help managers, researchers, and policymakers arrive at conclusions about the contribution their program has made (or is currently making) to particular outcomes, are useful here.

Emergent nature of foresight initiatives

Much foresight work is, by definition, 'emergent' since the initiative's endpoint has not yet happened and is unknown. Emergence refers to the process whereby the existence or formation of novel collective behaviours, properties, or phenomena come into existence only when the parts of a system interact in a wider whole (Darling et al., 2018). For example, foresight workshops may engage participants in a co-creation process to imagine alternative futures that have not been envisioned by workshop facilitators before.

These sorts of initiatives require an evaluation approach based on a greater understanding of the role of emergence in social change. It's important to understand the role of emergence in foresight and consider approaches like developmental evaluation. Moreover, it's important to identify and document emergent partners, strategies, and outcomes rather than only paying attention to the objectives and targets identified at the beginning.

Likewise, it is important to identify the aspects of a foresight initiative that are not emergent and can be approached using conventional evaluation methods, such as assessing participant satisfaction, knowledge changes, and implementation fidelity.

Generalisability and validity

While determining attribution is a challenge in many foresight activities, evaluations can be designed to increase rigour, including collecting baseline data for a pre/post or longitudinal analysis, triangulation of data from diverse sources—interviews, site visits, surveys, financial analyses--to corroborate (or refute) findings, having a reasonable sample size and response rate in surveys, identifying a counterfactual (such as survey with placebo technique), and inquiring about unintended consequences.

Useful evaluation concepts and tools

In this section, we highlight those evaluation approaches and methods that are well suited for evaluating foresight, addressing the challenges noted above and providing evaluation findings that focus on process and impact (listed alphabetically) (Gardner et al., 2024).

Adaptive management

Foresight work can be uncertain both in outputs and the process. Adaptive management refers to adaptation that goes beyond the usual adaptation involved in good management or modifying plans in response to changes in circumstances or understanding and using information to inform these decisions. It is an approach to managing under conditions of ongoing uncertainty that represents a paradigm shift from classic, linear approaches to planning, implementation, and evaluation. It is well-suited for the emergent aspects of foresight activities, such as alternative scenario projects with unknown outputs (Rogers & Macfarlan, 2020).

Advocacy and Policy Change (APC) Evaluation

Advocacy and Policy Change (APC) Evaluation is an arena that has been developing its own unique methods since the early 2000s. This area is particularly germane to foresight activities that target decision-makers, civic engagement activities, and policy-supporting future generations. The field has produced many outcomes, methods, metrics, and tailored evaluation toolkits that combine constructs and methods useful for assessing public sector foresight and participatory foresight  (Gardner & Brindis, 2017).

Appreciative Inquiry

Since much foresight work focuses on preferred images of the future, it is helpful to work with evaluation methods that are compatible with the aspirational aspects of foresight. Appreciative inquiry is an approach to organisational change that focuses on strengths rather than weaknesses. Appreciative Inquiry is often presented in terms of a 4-step process around an affirmative topic choice:

1. Discover: What gives life? What is the best?  Appreciating and identifying processes that work well. 
2. Dream: What might be? What is the world calling for? Envisioning results and how things might work well in the future.
3. Design: What should be--the ideal? Co-constructing - planning and prioritising processes that would work well. 
4. Destiny (or Deliver): How to empower, learn and adjust/improvise (Coughlin & Preskill, 2003)?

Complex adaptive systems (CAS)

As described above, foresight work is complex and focuses on complexity. Complex adaptive systems (CAS) can be helpful for characterising and working with this complexity in an evaluation design. A complex adaptive system is a system that is complex in that it is a dynamic network of interactions, but the behaviour of the ensemble may not be predictable according to the behaviour of the components. It is adaptive in that the individual and behaviour mutate and self-organize, corresponding to the change-initiating micro-event or collection of events (Sibthorpe et al., 2004).

Contribution Analysis

Since much foresight work produces outputs, like alternative scenarios, that are set out in the future, determining attribution is challenging. Contribution Analysis is an approach for assessing causal questions and inferring causality in real-life program evaluations. It offers a step-by-step approach designed to help managers, researchers, and policymakers arrive at conclusions about the contribution their program has made (or is currently making) to particular outcomes. It is particularly useful in situations where the programme is not experimental (Mayne, 2012).

Developmental Evaluation

Another approach for working with the emergent and complex nature of foresight work is Developmental Evaluation. This approach can assist social innovators in developing social change initiatives in complex or uncertain environments. DE originators liken their approach to the role of research and development in the private sector product development process because it facilitates real-time, or close to real-time, feedback to program staff, thus facilitating a continuous development loop (Patton, 2011).

Evaluability Assessment

Before launching a foresight evaluation, evaluators should undertake an evaluability assessment (EA) and determine how ready a foresight activity is for evaluation. Evaluability Assessment assesses the extent to which an intervention can be evaluated reliably and credibly. The results of an EA should indicate what preparatory steps are needed to make an intervention ready for an evaluation and what remaining challenges will need to be addressed by an evaluation team. An EA will examine (a) the adequacy of the intervention's theory of change, (b) the availability of relevant data and supporting systems, (c) stakeholders' interests in the evaluation, especially their evaluation questions, and (d) constraints arising from the surrounding institutional context (Davies, 2013). 

Evaluative thinking

As a new area of learning and inquiry, evaluators of foresight activities should take time to reflect on their evaluation practice and its strengths and weaknesses. Evaluative thinking is a mindset that goes beyond the perfunctory assessment of a program's value and involves: “identifying assumptions, posting thoughtful questions, pursuing deeper understanding through reflection and perspective taking, and making informed decisions and preparation for action” (Vo et al., 2015). Not only will this benefit the evaluator, but foresight evaluation participants will also have a willing thought partner who can bring useful tools, such as working with a theory of change, to understand a foresight initiative better and its possible impacts.

Monitoring

Foresight practitioners not only pay attention to project outcomes and impacts but are also interested in how well their project or workshop is being implemented and areas for improvement. Monitoring is a process of periodically collecting, analysing, and using information to actively manage performance, maximise positive impacts, and minimise the risk of adverse impacts. It is an important part of effective management because it can provide early and ongoing information to help shape implementation before evaluations (Markiewicz, 2014).

Most Significant Change approach

Because foresight focuses on change in its many forms, methods that characterise and prioritise change in a particular setting are very relevant. The Most Significant Change technique is a form of participatory impact monitoring and participatory evaluation, which involves soliciting and analysing personal accounts of change and deciding which of these accounts is the most significant – and why. Learning opportunities occur at two levels: (a) the choices of which types of change are more versus less preferred, i.e. the direction of change, and (b) the choice of what criteria of value are most important in a given context, i.e. the nature of the objectives being pursued by those changes. In addition, how MSC story selection processes are structured enables differences and similarities in views of different stakeholder groups in a foresight process to be more visible (relating to both levels of learning) (Davies & Dart, 2005). Source: MandE NEWS

Systems thinking

Foresight is considered a type of systems thinking since it examines the relationships between drivers, trends, events, and signals of change. Also, some foresight methods are systems thinking tools, such as futures wheels, allowing stakeholders to look quickly at the consequences of trends and events. While systems thinking evaluation tools are still being developed, it is helpful to have some competence in systems thinking concepts, particularly the distinction between hard and soft systems and concepts from systems dynamics, such as reinforcing and balancing feedback loops (Gates et al., 2021). Also, logic models and theories of change have many of the same attributes as systems thinking and have immediate practical benefits.

Transformational evaluation

Like systems thinking, foresight can support the call for evaluation to strengthen transformational change, putting new demands on evaluators to be flexible in the face of change and adopt new ways of thinking. Foresight methods such as the futures wheel, which explores future implications of trends, events, and emerging issues, can support transformed evaluation: "'evaluation must consider all interventions in their broader context and how they interact with human and natural systems', given their significant impacts. Evaluation must also move beyond focusing on individual projects and their stated objectives to consider their impact on wider systems,” (Van den Berg, 2021).