This paper provides detailed guidance on using big data to fill data gaps in impact evaluations. Data gaps can arise due to the inaccessibility of target populations, inadequate aggregation of data, data collection lag times, and data being missing in some contexts, like pandemics, conflicts, and humanitarian emergency situations. The paper includes a number of specific examples and additional references.
One issue to note when using the paper is that it uses the terms 'control group' and 'comparison group' inconsistently. The examples provided mostly refer to the use of comparison groups and quasi-experimental designs, not to randomly assigned control groups
This resource and the following information was contributed by Radhika Menon.
Authors and their affiliation
Authors - Rathinam, F., Thissen, P. and Gaarder, M.
Year of publication
Type of resource
This CEDIL Methods Brief takes a step-by-step, practical approach to guide researchers designing impact evaluations based on big data. This brief is based on the CEDIL Methods Working Paper on ‘Using big data for evaluating development outcomes: a systematic map’.
Who is this resource useful for?
- Advocates for evaluation;
- Evaluation users;
- Those involved in evaluation capacity strengthening.
How have you used or intend on using this resource?
This brief can enable researchers with some points they should consider when designing impact evaluations based on big data, although not all the steps will be relevant to every evaluation.
Why would you recommend it to other people?
The amount of big data available has exploded with recent innovations in satellites, sensors, mobile devices, call detail records, social media applications, and digital business records. Big data offers great potential for examining whether programmes and policies work, particularly in contexts where traditional methods of data collection are challenging. During pandemics, conflicts, and humanitarian emergency situations, data collection can be challenging or even impossible. This brief can enable researchers with some points they should consider when designing impact evaluations based on big data.
Rathinam, F., Thissen, P. and Gaarder, M. 2021. ‘Using big data for impact evaluation’, CEDIL Methods Working Brief 2. Oxford: Centre of Excellence for Development Impact and Learning (CEDIL). Available at: https://doi.org/10.51744/CMB2