Identify the resources that can be used for the evaluation, including potentially the following:
- Funding to engage external individuals or organizations to design and/or conduct the evaluation or review the design and the final report
- Staff time to either conduct the evaluation or to manage an external contractor
- Time and goodwill of other stakeholders who will be involved in the evaluation – such as partner organizations, community members.
- Existing data
Identify any particular constraints for the evaluation such as:
- Short time before findings are needed to inform decisions
- Poor reputation of evaluation due to previous experiences
- Difficulties in engaging particular groups or in working collaboratively
- Missing baseline data
- Difficulties in observing or getting data about implementation or results – for example, when it is being implemented in remote locations, or in fragile, conflict-affected areas.
- Disagreement about what success looks like – for example:
- Disagreement about the overall goals – for example, is an early childhood program primarily about improving workforce participation of parents or about early learning of children?
- Disagreement about the criteria that should be used – for example, is good research technically very accurate or produced in time to inform an important decision? Is the goal to improve the average health and wellbeing in a community or to ensure everyone is above the minimum requirement?
- Diagreement about the standards that should be used - for example, is a 10% increase in published research a good result?
Do a estimate of the costs to collect and analyse the data, as well as the project management and reporting time needed. If available resources are not adequate for the design, adjust the design and/or resources. Some possible options for reducing costs are shown below.
|
Possible way of reducing costs |
Possible implications |
How to manage these risks |
|
Reduce the number of Key Evaluation Questions |
Evaluation might no longer meet the needs of the primary intended users |
Carefully prioritise the KEQs Review whether the evaluation is still worth doing |
|
Reduce sample sizes |
Reduced accuracy of estimates |
Check these will still be sufficiently credible and useful through data rehearsal using interval estimates |
|
Make more use of existing data |
Might mean that insufficiently accurate or relevant data are used. The cost savings might be minimal if they are not readily accessible. |
This is only appropriate when the relevance, quality and accessibility of the existing data is adequate – need to check this is the case before committing to use them |
|
Embed data collection in program implementation |
Might lead to a reduction in data quality |
Ensure staff are trained and motivated to collect data properly and have sufficient time and equipment to do so |
|
Use fewer waves of data collection, including possibly retrospectively created baselines |
Will increase the risk of inaccurate data |
Check that retrospective baselines will be sufficiently accurate and that less frequent information on progress will be sufficient to inform decisions |
You are currently here:
+ - 2. Scope the evaluation
- Clarify what will be evaluated
- Describe the theory of change
- Identify who are the primary intended users of the evaluation and what will they use it for
- Develop agreed key evaluation questions
- Decide the timing of the evaluation
- Decide whether the evaluation will be done by an external team, an internal team or a hybrid of both
- Determine the evaluator qualities
- Identify what resources are available for the evaluation and what will be needed
Comments
There are currently no comments. Be the first to comment on this page!
Add new comment
Login Login and comment as BetterEvaluation member or simply fill out the fields below.