The evaluation report by the Abdul Latif Jameel Poverty Action Lab (JPAL) is an example of how the randomized control trial (RCT) evaluation methodology can be used to test impacts of communication on behaviour and social change. In this case, the impact of civic education programs on voter and politician behavior were assessed. When compared with other civic education efforts, JPAL found evidence that SFCG’s mobile cinema increased voter knowledge of candidates, and increased policy-based (over personality based) voting intentions. This evaluation also found that these debate screenings enhanced the accountability of elected officials to their constituents following the elections. It is an example of evaluation tasks relating to investigating causal attribution and contribution, in particular: Compare results to a counterfactual (strategy 1). A written report and a video are available.
The Evaluating C4D Resource Hub sits within BetterEvaluation and houses a growing collection of the available guides, toolkits, tools and methods to use for research monitoring and evaluation (R,M&E) of Communication for Development (C4D) initiatives. The Hub is structured around two combined frameworks:
The BetterEvaluation Rainbow Framework (represented by the rainbows) is a structure. It organises the practical tasks into seven categories or 'clusters' and provides options.
While the resource recommendation below discusses the resource specifically in relation to its usefulness for evaluating C4D within the Evaluating C4D Resource Hub's C4D Framework, this resource may also be of use for people working in other contexts and with different frameworks.
Authors and their affiliation
Kelly Bidwell, Katherine Casey, Rachel Glennerster
Year of publication
Type of resource
One of the interesting features of this case is the way that the nature of the 'roadshow' or travelling cinema is used to create treatment and control groups for comparison (discussed on page 7-8). This is a model that could be adapted to design an RCT-style investigation for a range of communication initiatives.
Who is this resource useful for?
- Communication Officers
How have you used or intend on using this resource?
This resource has been identified as particularly useful for evaluating media and communication initiatives, including communication for development (C4D). It could be adapted as a model for initiatives that include behaviour change objectives among both citizens and politicians where an experimental or quasi-experimental design is being considered. It was identified as part of a research project in collaboration with UNICEF C4D.
Why would you recommend it to other people?
This report would be valuable for teams considering options investigating causal attribution and contribution, and in particular those who are interested in using experimental or quasi-experimental designs to assess the impact of a program. It provides a model for making the most in differences between exposed and non-exposed groups based on the geographic locations of the 'roadshow' or travelling cinema. This example is consistent with the C4D Evaluation Framework in the following ways:
- accountable: quasi-experimental designs provide quantitative data, which is a valued form of evidence in development agencies to show that a program works and make a case for further investment.
On the other hand, the C4D Evaluation Framework would suggest the need to consider the following issues:
- complexity: as with all experimental and quasi-experimental designs, this creation of a counterfactual in the design of the research initiative required standardised implementation, and therefore did not allow the flexibility for adaptive and emergent approaches to C4D to be used. See other options for investigating causal attribution and contribution.
- participatory: experimental and quasi-experimental designs are generally not associated with participatory approaches, due to the need for standardisation and specific technical expertise. See other options for investigating causal attribution and contribution.
Bidwell, K., Glennerster, R. and Parry, K., 2015. DEBATES The Impact of Voter Knowledge Initiatives in Sierra Leone.