What is it?
In some cases, it might be possible to gather data on an entire population (for example, some data might be available from every participant, or about every project), but in most cases, it will be necessary to take a sample of projects, sites, events, or people. Deciding on sampling strategies is an important part of an R,M&E design. Decision should be strategic and well considered, informed by the purpose, the nature of the initiative, the nature and requirements of particular methods, and the resources available. Three broad types of sampling are: random sampling (which uses random or quasi-random methods to select the sample and then uses statistical inference to draw conclusions about the population); purposeful sampling (which selects information-rich cases to study and then use analytical inference to draw conclusions with wider applicability; and convenience sampling (which selects readily accessible cases and is at greatest risk of bias).
More information on the sampling options is available on the main BetterEvaluation site here. This page is recommended background reading before considering options to apply to C4D. Sampling should be considered alongside issues of response rate and coverage - results are more accurate from a well-chosen sample with a high response rate than from a population with a poor response rate that usually is biased.
Sampling and C4D
Applying the C4D principles
Situations that influence sampling decisions:
|Samples should include multiple perspectives, to understand differences in experiences in different settings. Complex interventions might need sampling strategies that can be adapted to suit emerging issues and understandings, such as using ‘purposeful’ sampling (selecting based on what is useful or most interesting) to follow up emerging patterns and findings.|
|Thoughtful and thorough sampling helps to make the R,M&E design more rigourous. In quantitative (numbers based) methods sampling the sample size and the sample selection are key to making credible claims about the findings. In qualitative (words, stories,visual) methods, sharing details about the sample and selection process increases credibility and trustworthiness.|
|More generally, sampling should pay attention to equity dimensions, and ensure that the most vulnerable groups are represented and that the data is able to be disaggregated. Additional effort might be needed to get adequate coverage of more remote, more disadvantaged groups due to known biases such as: roadside bias, seasonal bias, pro-literacy bias, etc.|
Recommended options and adaptations for C4D
Doing Qualitative Field Research on Gender Norms with Adolescent Girls and Their Families is a guide to conducting qualitative research with children and young people with a focus on gender sensitivity. It includes useful advice relevant to sampling such as 'Include girls from different backgrounds and in different situations', 'Include girls’ parents, grandparents, siblings and other family and community members' and 'Think through which issues are priorities for you to explore' (p4). To read more about how this resource is consistent with the C4D Evaluation Framework. This guide is consistent with the C4D Evaluation Framework in relation to this task in the following ways:
- holistic: the guidance on sampling suggests thinking about cases as samples, including other people around the girl who influence her context.
- complex: the guidance focuses on rich descriptions from multiple perspectives, rather than the number of respondents
- realistic: the guide advises choosing a sample that helps achieve the objectives
- critical: the guide suggests seeking out girls with different kinds of experiences and backgrounds.
How to Determine a Sample Size: Tipsheet #60 (2001) - this is a clear and accessible guide to making decisions about samples in quantitative data, such as surveys (for example, Knowledge, Attitudes and Practices surveys). It is consistent with the C4D Evaluation Framework in the following ways:
- realistic: the steps in the tip sheet move back and forth between the resources available and the desired precision level and risk.
- critical: the final tip points to the fact that characteristics of non-respondents are significantly different, and suggests that understanding the limitations of the method and using mixed methods is important.