A handbook of data collection tools

This handbook, written by Jane Reisman, Anne Gienapp and Sarah Stachowiak for the Annie E. Casey Foundation, is a companion to A guide to measuring advocacy and policy. The handbook includes a range of practical tools and processes that can be used to gather data about policy and advocacy initiatives.  

Excerpt

"These examples are actual or modified tools used for evaluating existing campaigns or related efforts. We aimed to identify a wide range of data collection methods rather than rely primarily on traditional pre/post surveys and wide opinion polling. When possible, we included innovative applications of tools or methods to provide a broad range of options for grantees and funders.

We primarily identified sample tools to measure the core outcome areas related to social change or policy change. For each outcome area, you will find several data collection options as well as relevant methodological notes on ways to implement or adapt particular methods. In addition, we have included examples of tools and methods related to other types of evaluation  design." (Reisman, Gienapp and Stachowiak, 2007)

Contents

• Measuring Core Outcome Areas 1
  • Outcome Area: Shift in Social Norms   2
    • Interview Protocol: Changes in Awareness and Prioritization  2
    • Focus Group: Changes in Attitudes   4
    • Meeting Observation Checklist: Changes in Community Members’
    • Beliefs about the Importance of a Particular Issue   4
    • Survey: Changes in Prioritization of Specific Issues  6
    • Rolling Sample Survey: Changes in Community Awareness  6
  • Outcome Area: Strengthened Organizational Capacity   7
    • Self-Assessment Tool: Alliance for Justice Advocacy Capacity
    • Assessment   7
    • Self-Assessment: Spider Diagram   9
    • Self-Assessment: KIDS COUNT Self-Assessment Tool 10
  • Outcome Area: Strengthened Alliances 12
• Tracking Form: Intensity of Integration Assessment  12
  • Outcome Area: Strengthened Base of Support 14
    • Tools for Measuring Public Support 15
    • Logs: Increased Public Involvement in an Issue 15
    • Log: Increased Engagement of Champions  15
    • Survey: Increased Public Involvement 16
    • Self-Assessment: Checklist for Mobilization and Advocacy  17
    • Tools for Measuring Media Support  20
    • Media Tracking Form: Increased Media Coverage  21
    • Composite News Scores: Media Impact 23
    • Log: Increased Visibility  23
  • Outcome Area: Improved Policies 24
    • Log: Legislative Process Tracking  26
    • Log: Policy Tracking Analysis 26
    • Survey: Assessing Number and Type of Policies 28
    • Log: Monitoring Policy Implementation  28
    • Environmental Assessments: Changes in Physical Environments  30
  • Outcome Area: Changes in Impact  32
• Tools and Methods for Other Evaluation Designs  33
• Evaluation of Strategic Progress  33
  • Method: Appreciate Inquiry Approach to Process Evaluation 34
• Measuring Short-Term Incremental Objectives  35
• Case Study Documentation of Process and Impacts  38
  • Method: Case Studies 38
  • Method: Theory-based Evaluation  40
  • Method: Use of Coding Protocol with Qualitative Data  42
  • Reporting Tool: Alliance for Justice Advocacy Evaluation Tool  45