C4D: Generalise findings

What is it?

An evaluation usually involves some kind of generalisation of the findings to put forward an analysis that predicts how the findings of one initiative might relate to future programs, other places and contexts, or other groups of people. Often it is assumed that statistical generalisation is the only way to generalise, but there are a range of methods for undertaking this task.

General information

The Rainbow Framework includes comprehensive guidance on methods with links to resources, including both statistical and non-statistical generalisation methods. This page is recommended as background reading before considering methods to apply to C4D.

Applying the C4D principles to generalising findings from C4D

Complexity

Although there may rarely be a one-size-fits-all set of recommendations for C4D, there may be some key principles or insights about the kinds of contextual factors that have the most influence and can be generalised.

Participatory

The knowledge of partners, communities and other stakeholders can be valuable in drawing out key principles or insights that can be used to consider whether the same initiative might work in other contexts (other times, places and people).

Critical

Consider who the initiative has worked for and where (who has it not worked for) and how this might this translate to other contexts (places, people and groups). When using participatory approaches to generalising findings, consider whose perspectives are included and silenced in this process.

Holistic

When Generalising Findings it is important to identify what the key social, political, economic, cultural and other systemic factors were, in that specific place and time, that affected whether it worked. This will help to predict what factors will need to be considered in other contexts.

Recommended methods and adaptations for generalising findings in C4D

Approaches

Realist evaluation
Realist Evaluation is a complete approach to evaluation, however, it is also possible to just borrow the key concepts relating to causality and generalisation for this task. The realist evaluation approach stresses the importance of context in understanding causes and begins from the premise that causal mechanisms will only lead to those causes when the context is conducive. Therefore, claims about generalisation of findings are usually modest and contingent. Instead, it seeks to provide plausible explanations of what happened and why, with a focus on the conditions that made the changes possible. It is this focus on the conditions and contexts that can help inform assessments of whether interventions that proved successful in one setting may be so in another setting (often another specific setting, rather than an abstract or hypothetical setting)

This approach is consistent with the C4D Evaluation Framework in the following ways:
- Critical: realist evaluation is always sensitive to differences, asking not just 'what has worked' and but also 'for whom'.
- Holistic: realist evaluation is highly sensitive to context and conditions, asking not just 'what has worked' but 'what has worked in what circumstances'. The conditions that support the change to happen are a key part of any assessment of generalisability.
- Complex: a realist evaluation approach can help make sense of the complex processes underlying programmes by formulating plausible explanations
Positive deviance
The Positive Deviance approach treats generalisability in a slightly different way. Investigators work with communities using participatory approaches to identify outliers to the norm; people or groups who stand out as positive cases, deviating from the general trends. The Positive Deviant approach then seeks to 'discover' the uncommon behaviours and strategies that led to better solutions to problems. This informs a “Design” of initiatives to make more widespread (or 'scale up') the use of solutions through iterative processes. This approach is consistent with the C4D Evaluation Framework in the following ways:
- Participatory: the approach is premised on the belief that communities already have the expertise and solutions to solve their own problems, participatory and community-driven approaches to discovering and analysing these are key.
- Learning-based: positive deviance treats generalisability of solutions from one positive case as a goal that can be achieved through iterative and action-oriented processes to test and assess solutions.
- Complex: the positive deviance approach is premised on the idea that communities are self-organising. The process requires highly adaptive approaches and comfort with unpredictability.
Horizontal evaluation
Horizontal Evaluation treats generalisability differently again. In this approach, peer-learning and peer-evaluation between different groups doing similar kinds of initiatives is a mechanism for encouraging those participants to adapt and apply successful approaches. This approach is consistent with the C4D Evaluation Framework in the following ways:
- Learning-based: Horizontal Evaluation approaches generalisability as an outcome of peer-learning, where one of the main objectives is to learn and adapt good practices by peers.
- Participatory: rather than 'expert led' the horizontal evaluation approach uses participatory processes toward peer evaluation.
- Complex: the Horizontal Evaluation approach depends on the self-organising capacity of participants to recognise aspects that can be adapted and generalised to their own context
- Critical: The involvement of peers overcomes some of the uneven power relations that can occur in external evaluations, however it is important to have an experienced facilitator who can create a trusting environment and ensure participation of all people.

Resource

The community radio continuous improvement toolkit

This toolkit is premised on a mix of self-assessment and peer-assessment toward co-learning. It was created in the context of community radios in India, but, with some adaptation of the questions, the processes and guidance could be applied to support peer-assessment between organisations doing a range of different types of C4D.