Seven evaluation methods to add to the toolbox?
In a recent workshop on 'Designs for Performance Evaluation', which Patricia Rogers conducted with program officers from USAID, we looked at seven methods and strategies which might be usefully added to the repertoire for collecting, analyzing or synthesizing data.
Which of these might be useful to add to the methods that you use - or that you encourage evaluators to use in evaluations that you manage?
1. Mobile Phone Data Logging
Mobile phones (cell phones) can be a great method for collecting data - and they don't have to be smart phones. Phones can use SMS, Java or Android to log data. Advantages of using this method include making decentralized data collection easier and providing immediate reporting and analysis.
For an example, check out how Water for People is using a mobile phone-based technology called FLOW – Field Level Operations Watch – to maintain accountability and allow communities to monitor their own water and sanitation.
NOMAD has developed a useful review of the current state of the art in mobile data collection systems
2. Community mapping of an area’s resources and problems
Community mapping can be used to aggregate local citizen reports, community media and other relevant news and information.
The Map Kibera project has gathered data about the location of services to produce the first map of Kibera, Nairobi (one of the largest slums in Africa).
Map Kibera http://mapkibera.org/
3. Interactive mapping of crowdsourced data
Interactive maps can be built from data contributed by text messages, email, twitter, news and web-forms. Like mobile phone data logging this method allows for decentralized data collection and rapid reporting and analysis.
Ushahidi is a non-profit tech company that specializes in developing free and open source software for information collection, visualization and interactive mapping.
Stop Stockouts uses interactive mapping to share information and advocate against stockouts (when a pharmacy has no medicine on the shelf).
Stop Stockouts map of pharmacies out of stock of first line malaria drugs
4. Photovoice
This method involves community members in taking photos, selecting the most significant photos, telling stories about what they mean, and identifying key themes.
PhotoVoice builds skills within disadvantaged and marginalised communities using participatory photography and digital storytelling methods to represent themselves to achieve positive social change.
5. Structured observation
Instead of having vague and unsystematic observation during field visits, this method uses trained and experienced observers who look for particular aspects of implementation and impact, record what they observe where possible, record details about the context, and check the meaning of their observations.
Useful guidance on observation can be found in Tips on Observation from the USAID Center for Development Information and Evaluation.
6. Keypad technology
This technology allows data to be collected anonymously from individuals in a group setting using special clickers or texts from cellphones. Uses can select from multiple choice options and the results can be displayed immediately. The results can form the basis for further discussion or be used as voting.
The National Coalition for Dialogue & Deliberation has developed information on keypad technology .
7. Evaluation rubric
A presentation by Jane Davidson, Nan Wehipeihana and Kate McKegg describes evaluation rubrics and two examples of their application.
Read more:
Case study by Judy Oakden on the application of rubrics in an education program
Update: We updated this post on 18 September 2014 to add links to resources on the BetterEvaluation site that have been added since the post was first published.