Decide whether the evaluation will be done by an external team, an internal team or a hybrid of both

Expertise, impartiality, cost, and time are key issues in deciding who will conduct the evaluation.

The table below lists specific trade-offs in the decision-making about engaging internal or external evaluators:

  Internal evaluator(s) External evaluator(s)

Perspective

May be more familiar with the community, issues and constraints, data sources, and resources associated with the project/program (i.e., insider perspective).

May bring fresh perspective, insight, broader experience, and recent state-of-the-art knowledge (i.e., outsider perspective).

 

Knowledge and skills

Are familiar with the substance and context of research for development programming.

May possess knowledge and skills that internal evaluators are lacking but it may be difficult to find evaluators who understand the specifics of research for development programming.

Buy-in

May be more familiar with the project/ program staff and may be perceived as less threatening, and, thus, may have greater buy-in and staff involvement in the evaluation.

May be perceived intrusive or a threat to the project/program (i.e., perceived as an adversary) and, thus, may have more difficulty obtaining relevant information.

Stake in the evaluation

May not be seen as an honest broker but may be perceived as having an agenda / stake in the evaluation.

Can serve more easily as an arbitrator or facilitator between stakeholders as perceived as neutral.

Credibility

May be perceived as biased when perceived as ‘too close’ to the subject matter which may result in lesser credibility of the evaluation hindering its use.

May provide a view of the project/program that is considered more objective, not part of the organization’s power structure, and thus, give the findings more credibility and potential for use.

Resources

May use considerable staff time which is always in limited supply, especially when their time is not solely dedicated to the evaluation.

May be more costly and still involve substantial management time from the commissioning organization’s staff.

Follow-up / Use of evaluation findings

More opportunity and authority to follow up on recommendations of the evaluation.

Contracts often end with the delivery of the final product, typically the final evaluation report which limits or prohibits follow-up.

As outsiders, do not have authority to require appropriate follow-up or action.

It is advisable to engage an external evaluator / evaluation team when:

  • the scope and/or complexity of the evaluation demand expertise that is not internally available;
  • a program or project is politically sensitive and impartiality is a key concern; or,
  • internal staff resources are scarce and timeframes are particularly pressing (i.e., there is little flexibility in terms of evaluation timing).

External evaluators may be an individual, a research institute or a consulting firm.

Planning and executing an evaluation is, in any case, a team effort. A critical decision to be made at this stage is who will lead the evaluation. Consider the following options for assembling the evaluation team:

  • External evaluator(s) –one of them serving as the team leader– supported by program staff
  • Internal evaluator(s) –one of them serving as the team leader– supported by program staff
  • An internal evaluator –serving as the team leader– supported by other internal evaluators and program staff but also external evaluator(s)

Even if an external evaluator is hired to conduct the evaluation, the program manager and other staff must be involved in the evaluation process. Staff are not only primary users of the evaluation but also participants in data collection (such as providing access to records, educating the evaluator about the project/program or being interviewed as a key informant) and/or other evaluation-related tasks. Be realistic about the amount of time needed for this involvement so staff schedules do not get over-burdened.

Although hiring an external evaluator may seem costly, it may, ultimately, be less expensive than channeling considerable staff time into the evaluation. A careful analysis of staff time costs compared to external consultant costs is needed before making a decision.

For partnership evaluations (i.e., co-funded by more than one organization), double-check whether any of the co-funders have a requirement for using a ‘third party’ evaluator (i.e., someone who is not affiliated, in any way, with any of the organizations involved) (see also Step 1).

Further information & Resources

IDRC-specific information: 
If you are an IDRC member, please login to see information specific to IDRC.
You are currently here:
Click here to view full menu

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.