Week 45: Tracking support among U.S. members of congress
Scorecards are used in many different types of evaluation, and can have influence through informing decisions and by making performance visible. This week's guest blogger, Jennie Aylward, describes a score card used to report on an advocacy program aimed at members of the U.S. Congress.
The current U.S. Congress presents a particular challenge to advocates focused on U.S. policy: it is on track to be the least productive Congress since World War II, in terms of legislation passing. When almost nothing for which you advocate ever becomes law, how do you know you’re making a difference on Capitol Hill?
CARE USA, a non-profit organization, has noted that “policy” is not comprised solely of laws, and policymakers can demonstrate support for a cause in a myriad of ways. In CARE’s case, that cause is promoting the rights and well-being of marginalized people around the world.
Over the years, CARE has guided over 100 Members of Congress on learning tours to family planning programs, food aid reform initiatives, and programs to prevent gender-based violence. The learning tours are intended to inspire Members to seek policy change and preserve U.S. funding for foreign assistance, and give them the background to ensure the policies they pursue are effective.
CARE staff wanted to evaluate the degree to which Members of Congress function as champions for CARE-supported policy positions after they visit CARE sites abroad, so they pioneered an internal M&E “scorecard.” While the scorecard concept is a familiar one in U.S. policy circles (consider those employed by the National Rifle Association and NARAL Pro-Choice, to offer two examples), CARE’s version is much more nuanced than a simple voting record, and is primarily for the organization’s internal use.
CARE staff identified criteria against which to evaluate Members’ level of activism, and assigned weights to each activity. A Member would receive a higher score, for example, by organizing a congressional hearing that supports CARE’s policy objectives than by simply attending such a hearing and posing questions to a witness in ways that reflect CARE’s policy interests – though both activities are valuable. Every possible such activity is featured in the scorecard with a point value, and that point value is multiplied by the number of times the Member engages in the activity.
Sarah Hemingway Lynch, Senior Director of Global Policy Initiatives at CARE USA, oversees the scorecard initiative. “It’s subjective,” she noted of the scoring system, “but it’s a place to start” understanding the impact of the Member’s trip, and it also makes clear CARE’s ideals for Members’ activities as champions.
CARE has partnered on the scorecard initiative with the Aspen Institute’s Aspen Planning and Evaluation Program (APEP), which combines CARE’s information about its interactions with Members of Congress with open-source records of Members’ activities, such as congressional hearing transcripts and media reports. David Devlin-Foltz, APEP’s Executive Director, explained that the public records of Member activities are key to the utility of the scorecard – the actions of other types of potential advocacy targets (such as celebrities or lower-level officials) are not as well-documented, nor do those targets necessarily have uniform “champion” options available to them, such as holding a hearing.
APEP quantifies the information from various sources, and presents it in various visual forms. For example, one product of the scorecard is a stacked graph that shows a Member’s level of activism before their trip with CARE, and over time after they return – 1-4 months afterward, 5-8 months afterward, etc.
CARE staff use the scorecard and its visual products to see trends in a Member’s activities. By looking at the differences in type and level of engagement they have had with more and less activist Members, CARE staff can also identify why some Members are more active champions than others, and reorient their tactics toward those that inspire the strongest Member responses.
While it is primarily an internal tool, CARE has learned that Members may be moved to action by seeing their own scorecards. Sarah described a Congresswoman who had seen family planning programming during her travels with CARE. In the immediate aftermath of her trip, she had been a very active supporter, but the support dropped off after that. CARE took her scorecard to her Chief of Staff, and asked her to consider the Congresswoman’s trajectory of support, as illustrated by the scorecard’s bar charts and bubble graphs. CARE’s message was positive: the Congresswoman was clearly moved by the issue, and she could have a positive policy impact if she resumed her role as a champion. The scorecard visuals “really got through” to the Chief of Staff, and the Congresswoman went on to co-sponsor a relevant bill.
Given the current legislative climate in Washington, that bill may never move forward, but the example illustrates what can be achieved with an M&E tool like CARE’s scorecard. Instituting a fairly straightforward system to track and demonstrate the impact of advocacy activities has allowed CARE to target its advocacy better, and thereby move its policy agenda forward.
Resources
Guides
Community Score Card Process - A Short Note on the General Methodology for Implementation
This guide outlines the Community Score Card Process and details its four components. It uses tables and figures to examine the CSC Process, and also uses examples to demonstrate the different types of Community Score Cards that are available.
Community Mapping: Community Scorecard Process Guide
This 32 slide powerpoint presentation, developed by the Participation and Civic Engagement Group, Social Development Department, of the World Bank introduces the concept and methodology of the community scorecard process. Read more
Examples
Community Scorecard Approach for Performance Assessment - ProNet North's Experience
This WaterAid Ghana Briefing Paper, outlines ProNet North’s experience of using the Community Scorecard Approach to assess performance.
2012 California County Scorecard of Children's Well-Being
This website from Children Now provides an interactive display of statistics reporting on the wellbeing of children in California. It uses 28 indicators to rank child well-being in each of California's 58 Counties.
Development theme
Evaluating Policy Influence and Advocacy
Influencing and informing policy is the main aim for many development organisations. However, activities directed at policy change are, in general, very hard to monitor and evaluate. As policy change is often a complex process, it is difficult to isolate the impact of a particular intervention from the influence of other factors and various actors. In addition, monitoring and evaluation tools usually used in managing interventions can be difficult to implement in these contexts.