C4D Hub: Collect and or retrieve data (methods)

What is it?

Data collection methods should be selected on the basis of how well they will answer the key questions, with due consideration of available resources. Decisions about methods need to be made in conjunction with other decisions about the key questions (what to collect data on), whether indicators might be used, how sampling will be used, and how data will managed and analysed. When decisions are made, these should be documented in Planning Documents. The data collection methods on this page will generate descriptive data: information about what has happened or how things are through measuring or describing things. 

General information

While there are many different methods for data collection/retrieval, they can be grouped into the following types: information from individuals (eg key informant interviews); information from groups (e.g. focus group discussions); observation (either directly or through photographs and videos, including aerial observation); physical measurements; and existing records and data (including social media and other media). General information on methods and other options is available on the Collect or Retrieve data page. This page is recommended background reading before considering options to apply to C4D. 

Collecting and/or retrieving data and C4D

Applying the C4D principles

Participatory 

Some methods are more engaging, less extractive, and enable mutual learning, to a greater extent than others.  Although this is not the full extent of what it means to take a participatory approach, methods of this nature are recommended. 

Holistic 

If your key questions set out to explore contextual factors, the methods you chose to answer the questions need to be the type that helps you construct 'thick descriptions' (comprehensive, in-depth, contextual).

Critical 

We need to be conscious of gender and other power inequalities that exclude marginalised groups from contributing to the process: analytically reflect on how methods may distort, exclude or silence particular perspectives and voices 

Realistic 

Choices about methods must remain practical, pragmatic, and feasible, and fit with the available resources. This may involve compromise to remain realistic, however, in C4D ensuring that local needs, voices and experiences are given prominence should remain a priority.

Complex 

Data methods should be chosen for how well they will show different perspectives and experiences, and increase understanding of how contextual factors influence outcomes. In complicated and complex interventions, quick methods (compared to slow methods like national surveys) will be more useful for informing adaptive implementation of C4D.

Recommended methods for collecting data to answer descriptive questions relating to C4D:

DATA FROM INDIVIDUALS OR GROUPS

Communicative Ecology Mapping

An interactive method which seeks to uncover rich details about communication environments, uses and contexts through mapping. They can be made with individuals or with groups of people during discussion or drawn up afterwards on the basis of discussions and then checked with the participants. It is particularly useful for C4D situation analysis and intervention design. It is consistent with the C4D Evaluation Framework in the following ways: 

HolisticCommunicative ecology mapping enables participants to share how they use different communication spaces, uses and contexts. It does not assume that communication looks the same in all places in the way that standardised surveys might. 

ParticipatoryCommunicative Ecology Mapping can be an engaging and visual method where participants map their communicative ecology (although it can also be created from interview data) 

Resources

  • Communicative Ecologies and Communicative Ecology Mapping is covered in the EAR Toolbox.

Examples

  • Communicative Ecology Mapping was used as part of an assessment of the Violence Against Children campaign in Vietnam. It mapped children's communicative ecologies, and was intended to be used for planning the next phase of the campaign. See Appendix page 52-54 

Participatory Rural Communication Appraisal

Participatory Rural Communication Appraisal was adapted from ‘Rapid Rural Appraisal’ (RRA) as a way to conduct multidisciplinary and participatory research in rural settings without requiring the intensive time commitment assumed by other qualitative investigations. By actively involving community members in the research process, the method also builds capacity by training people in research and involving them in the analysis. It is consistent with the C4D Evaluation Framework in the following ways:

ParticipatoryParticipatory Rural Communication Appraisal uses appropriate participatory techniques to involve participants in reflection and learning processes.

RealisticLike RRA, Participatory Rural Communication Appraisal focuses on 'rapid' and less time intensive participatory approaches to participatory research. 

Resources 

Examples

  • A study exploring Knowledge Attitudes and Practices relating to Violence Against Children in Tanzania used Participatory Appraisal (not necessarily PRCA), among other methods. Read more about this exemple here.

Critical Listening and Feedback Sessions (also Participatory Viewing and Listening)

Critical Listening (or viewing) and Feedback sessions is a process of group listening to or viewing content, followed by reflective and analytical discussions and responses. It has been used in C4D as a way for content producers to critically reflect on their content, and to get feedback from key community groups and audiences. This method is consistent with the C4D Framework in the following ways:

  • Learning-based: The feedback can be used to continually improve content, or as part of an assessment of content.
  • Holistic: The process enables rich explorations of meaning and interpretation of content.
  • Participatory: The process is a highly engaging method in which participants engage in mutual learning.

Resources:

Module 4 of the Equal Access Participatory Monitoring and Evaluation toolkit provides an overview of Critical Listening and Feedback Sessions.

Example

The Ruka Juu II Young Farmers in Business impact study (from 2013) is a good example of the use of participatory viewing and listening sessions undertaken with various community groups as part of an impact assessment. Click here to read more.

Surveys

Surveys and questionnaires are a set of structured questions that aim to collect specific information from the chosen respondents (written or orally). The questions are designed to gather information about attitudes, preferences and factual information of respondents, and can be useful when information from a representative sample is required. Knowledge Attitudes and Practices surveys (KAPs) are common in C4D. However, experience from practice suggests that KAP surveys are often limited and unsatisfactory for a deep understanding contexts and causes. It is consistent with the C4D Evaluation Framework in the following ways:

AccountableBecause a survey can collect data from greater numbers of people it helps to give a sense of the scale of impact, which is useful in discussions about effectiveness and impact.

CriticalSurveys should include some questions about demographics, which can be useful for disaggregating data and understanding differences and equity dimensions (see Sample).

Resources

Example

  • Tuzungumze na Watoto (T-Watoto) is an example of how a system for regular mobile phone household surveys can be set up by partnering with a local call-centre to regularly collect data for monitoring and evaluation. A representative sample of randomly selected households are surveyed, depending on the sample-size requirements. Any member of the household may be interviewed. 

Key Informant Interviews 

A Key Informant Interview (KII) involves gathering information directly from an individual who has good knowledge or experience on a subject of interest to the study or evaluation. KII is useful and effective when the person doing the interview is trusted by the key informant. This allows the interviewer to probe or ask further questions until he or she gets the necessary information. It is common method and can be consistent with all the C4D Evaluation Framework principles. In particular:

     Holisticsemi-structured interviews with key informants in particular allow for open exploration of points and factors

     Criticalpeople who may not be able to participate fully in group settings may feel more comfortable to speak in interviews, especially if the interviewer is trusted

Resources

Focus Group Discussions  

A Focus Group Discussion (FGD) is an effective way to capture information about norms, behaviours, practices and the variety of opinions or views within a particular population or group (e.g., adult married women, female teachers, and male farmers). The richness of focus group data emerges from the group dynamics and from the diversity of the group. FGDs may help identify commonly held views among group members, including – at times – divergent views. An FGD usually gathers 8 to 15 individuals (not too many) who represent a specific group to talk about a specific subject. The composition of the group is important: depending on the sociocultural setting, it may be inappropriate to host mixed groups (e.g., adolescent girls and boys). Further, age and gender are important considerations. Focus Group Discussions can be consistent with all the C4D Evaluation Framework principles. In particular:

     HolisticFocus group discussions allow for participants to drive open exploration of points and factors and discuss ideas together.

     Critical: In focus group discussions are usually conducted with groups of people that are similar or diverse (such as women, men, adolescents, community leaders, etc.), which helps to reduce barriers to participation, and enrich the quality of voices.

Resources

Examples

  • A study exploring Knowledge Attitudes and Practices relating to Violence Against Children in Tanzania used focus group discussions in a highly engaged, dialogical way, among other methods. Read more about this example here.

Social Mapping  

Social Mapping is a cartographic, two-dimensional, visual representation of the distribution of resources, services, processes, social relationships, and networks. Mapping may help to assess not only where key resources and places are located, but why certain services are or are not being accessed by all members of the community (e.g., why certain health clinics might not be visited by women or children). It can also be used to understand the organisation of institutions. A variant of Social Mapping, Body Mapping, can reveal people’s anatomical ideas and health concepts, aspects related to mental and physical health, wellbeing, and even child protection issues such as sexual abuse. It is consistent with the C4D Evaluation Framework in the following ways:

ParticipatoryThe visual and group-based nature of the method makes it a more engaging method where participants can actively lead the direction of discussion.

Critical: This is a critical method that allows for the decentralisation of power and control in the data collection process. 

HolisticBy moving away from interview techniques that are strictly guided by predetermined questionnaires or closed-ended questions, mapping and follow-up interviews can reveal cultural barriers, the beliefs that hold them in place, and bottlenecks that may have never arisen from traditional surveys or interviews.

Resources

Transect Walk 

The Transect Walk is a Participatory Rural Appraisal (PRA) tool for observing the terrain and everyday life in a given place from the perspective of local community members. During the walk, stops are made along the way, and observations are discussed with community members. After the walk is over, a small group discussion may ensue. Use in Equity-focused Monitoring Transect walks can help provide an overview of the distribution of resources, use of a particular service or supply, or other specific features of a settlement in a short period of time. For example, a transect walk may be used to check for treated bed-nets in every other house and asking persons in that house who sleeps under them. It is consistent with the C4D Evaluation Framework in the following ways:

Participatorythe group-based nature of the method makes it a more engaging method where participants can lead the direction of discussion.

Criticalwalks can be a powerful way of uncovering differences between groups.

Holistic: walks can reveal the interconnected nature of problems and change, and locate issues in the local environment and context. 

Resources

EXISTING DOCUMENTS

Media Review 

A media review is the process of studying newspaper articles, letters to the editor, television or radio broadcasts, possibly advertisements, and other types of media as applicable in order to understand the range of opinions around a specific issue of concern. It can be used to in the context of advocacy communication work. It is consistent with the C4D Evaluation Framework in the following ways:

RealisticThe bulk of the work in a media review is desk-based, and it can therefore be a less expensive option. 

Resources

Examples 

OBSERVATION

Participant Observation 

Participant Observation is a method used by ethnographic researchers while present in a community or organisational setting to gain a close understanding of people’s lives, including actions, interactions, behaviours and practices, through intensive involvement and participation, often over an extended period of time. The intention is that as participants become more comfortable and trust the researcher, the 'observer effect', where people change their behaviours because they know they are being watched, is reduced. Typically it is based on semi-structured and open-ended observation techniques, where extensive field notes are taken and where there is a flexible research design. It is consistent with the C4D Evaluation Framework in the following ways:

RealisticAlthough observation can lead to significant amounts of data in the form of fieldnotes (especially less structured, more open types of observation), it can be a less expensive option compared to interviews and focus groups.  

HolisticSometimes people's reported behaviour (in interviews, surveys, focus groups etc.) is different to their actual behaviour, where additional environmental, social and other factors can influence behaviour. Observation can help give different, more holistic insights.

Resources

Non-participant observation

Observing individuals and groups without actively participating or engaging. The observer takes on a more distant position and avoids influencing practices. However, it is important to recognise that even without active participation, people may change their behaviours if they know they are being observed (known as the observer effect). Non-participant observation may be structured (where very specific and pre-determined phenomena and variables are documented); semi-structured (where some areas of interest and variables may be pre-determined, with space to include additional details; or open ended (without any pre-determined structure). Examples include observation of hand washing practices among child and adult members of a community, observing a clinic session in a local health facility, or observing a community meeting where programme-related issues are discussed.

Additional Resources

Monitoring and Evaluation of Participatory Theatre for Change (PTC) sets out a logical process to develop a monitoring and evaluation plan with suggested tools for collecting data.  Click here to go directly to this resource.  The guide suggests the following methods: Community Assessment Scans (similar to Participatory Rural Communication Appraisal); Key Informant Interviews; and Focus Groups. This guide is consistent with the C4D Evaluation Framework in relation to this task in the following ways:

  • realisticthe guide suggests a good mix of methods which are fit for purpose, but will not become overly burdensome. The selection of tools is based on the questions, which are based on the Theory of Change. 
  • participatory the actors themselves are often included as part of data collection. This is especially the case for the community scans and the monitoring. 

Doing Qualitative Field Research on Gender Norms with Adolescent Girls and Their Families is a guide to conducting qualitative research with children and young people with a focus on gender sensitivity. Pages 7-9 provide useful advice on framing questions for adolescent girls in sensitive ways. Table 1 on page 10-12 includes a list of useful tools, purpose, who to include and why, key questions to ask, and tips on getting the most of out the tool. To read more about how this resource is consistent with the C4D Evaluation Framework. It covers general methods such as: Social Mapping (community mapping, body mapping); indepth interviews and key informant interviews; and Focus Group Discussions. It also includes guidance on gender-specific methods such as inter-generational trio (exploring social norms with three generations); marital network; outlier case study/life history. This guide is consistent with the C4D Evaluation Framework in relation to this task in the following ways:

  • holisticthe tools suggest vey open approaches to understanding contexts and factors, including a range of different people and perspectives. 
  • realisticthe guide is specific about the strengths of each tool, and offers very pragmatic advice for dealing with sensitive topics and situations.
  • criticalthe guide is sensitive to discomfort and sensitivities that the girls may be feeling and the suggests support both the research and the participants.

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.