In a recent workshop on 'Designs for Performance Evaluation', which Patricia Rogers conducted with program officers from USAID, we looked at seven methods and strategies which might be usefully added to the repertoire for collecting, analyzing or synthesizing data.
Which of these might be useful to add to the methods that you use - or that you encourage evaluators to use in evaluations that you manage?
1. Mobile Phone Data Logging
Mobile phones (cell phones) can be a great method for collecting data - and they don't have to be smart phones. Phones can use SMS, Java or Android to log data. Advantages of using this method include making decentralized data collection easier and providing immediate reporting and analysis.
For an example, check out how Water for People is using a mobile phone-based technology called FLOW – Field Level Operations Watch – to maintain accountability and allow communities to monitor their own water and sanitation.
NOMAD has developed a useful review of the current state of the art in mobile data collection systems
Mariel Lazcano using FLOW to interview a community member. Photo: Water for People
Read more: Mobile Data Collection page
2. Community mapping of an area’s resources and problems
The Map Kibera project has gathered data about the location of services to produce the first map of Kibera, Nairobi (one of the largest slums in Africa).
Map Kibera http://mapkibera.org/
3. Interactive mapping of crowdsourced data
Stop Stockouts uses interactive mapping to share information and advocate against stockouts (when a pharmacy has no medicine on the shelf).
Stop Stockouts map of pharmacies out of stock of first line malaria drugs
This method involves community members in taking photos, selecting the most significant photos, telling stories about what they mean, and identifying key themes.
PhotoVoice builds skills within disadvantaged and marginalised communities using participatory photography and digital storytelling methods to represent themselves to achieve positive social change.
Read more: PhotoVoice page
5. Structured observation
Instead of having vague and unsystematic observation during field visits, this method uses trained and experienced observers who look for particular aspects of implementation and impact, record what they observe where possible, record details about the context, and check the meaning of their observations.
Useful guidance on observation can be found in Tips on Observation from the USAID Center for Development Information and Evaluation.
- Field Trips: organizing trips where participants visit physical sites.
- Non-participant Observation: observing participants without actively participating.
- Participant Observation: identifying the attitudes and operation of a community by living within its environs.
- Photography/video: discerning changes that have taken place in the environment or activities of a community through the use of images taken over a period of time.
- Transect: gathering spatial data on an area by observing people, surroundings and resources while walking around the area or community.
6. Keypad technology
This technology allows data to be collected anonymously from individuals in a group setting using special clickers or texts from cellphones. Uses can select from multiple choice options and the results can be displayed immediately. The results can form the basis for further discussion or be used as voting.
Read more: Keypad technology page
7. Evaluation rubric
A presentation by Jane Davidson, Nan Wehipeihana and Kate McKegg describes evaluation rubrics and two examples of their application.
Case study by Judy Oakden on the application of rubrics in an education program
Update: We updated this post on 18 September 2014 to add links to resources on the BetterEvaluation site that have been added since the post was first published.