Consider the implications of the resources available and specific constraints

Identify the resources that can be used for the evaluation, including potentially the following:

  • Funding to engage external individuals or organizations to design and/or conduct the evaluation or review the design and the final report
  • Staff time to either conduct the evaluation or to manage an external contractor
  • Time and goodwill of other stakeholders who will be involved in the evaluation – such as  partner organizations, community members.
  • Existing data

Identify any particular constraints for the evaluation such as:

  • Short time before findings are needed to inform decisions
  • Poor reputation of evaluation due to previous experiences
  • Difficulties in engaging particular groups or in working collaboratively
  • Missing baseline data
  • Difficulties in observing or getting data about implementation or results – for example, when it is being implemented in remote locations, or in fragile, conflict-affected areas.
  • Disagreement  about what success looks like – for example:
    •  Disagreement about the overall goals – for example, is an early childhood program primarily about improving workforce participation of parents or about early learning of children?
    •  Disagreement about the criteria that should be used –  for example, is good research technically very accurate or produced in time to inform an important decision?  Is the goal to improve the average health and wellbeing in a community or to ensure everyone is above the minimum requirement?
    • Diagreement about the standards that should be used - for example, is a 10% increase in published research a good result?

Do a estimate of the costs to collect and analyse the data, as well as the project management and reporting time needed.  If available resources are not adequate for the design, adjust the design and/or resources.  Some possible options for reducing costs are shown below.

Possible way of reducing costs

Possible implications

How  to manage these risks

Reduce the number of Key Evaluation Questions

Evaluation might no longer meet the needs of the primary intended users

Carefully prioritise the KEQs

Review whether the evaluation is still worth doing

Reduce sample sizes

Reduced accuracy of estimates

Check these will still be sufficiently credible and useful through data rehearsal using interval estimates

Make more use of existing data

Might mean that insufficiently accurate or relevant data are used.  The cost savings might be minimal if they are not readily accessible.

This is only appropriate when the relevance, quality and accessibility of the existing data is adequate – need to check this is the case before committing to use them

Embed data collection in program implementation

Might lead to a reduction in data quality

Ensure staff are trained and motivated to collect data properly and have sufficient time and equipment to do so

Use fewer waves of data collection, including possibly retrospectively created baselines

Will increase the risk of inaccurate data

Check that retrospective baselines will be sufficiently accurate and that less frequent information on progress will be sufficient to inform decisions

You are currently here:
Click here to view full menu

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.