Decide whether the evaluation will be done by an external team, an internal team or a hybrid of both
Expertise, impartiality, cost, and time are key issues in deciding who will conduct the evaluation.
The section below lists specific trade-offs in the decision-making about engaging internal or external evaluators:
- Internal evaluator(s): May be more familiar with the community, issues and constraints, data sources, and resources associated with the project/program (i.e., insider perspective).
- External evaluator(s): May bring fresh perspective, insight, broader experience, and recent state-of-the-art knowledge (i.e., outsider perspective).
- Knowledge and skills
- Internal evaluator(s): Are familiar with the substance and context of research for development programming.
- External evaluator(s): May possess knowledge and skills that internal evaluators are lacking but it may be difficult to find evaluators who understand the specifics of research for development programming.
- Internal evaluator(s): May be more familiar with the project/ program staff and may be perceived as less threatening, and, thus, may have greater buy-in and staff involvement in the evaluation.
- External evaluator(s): May be perceived intrusive or a threat to the project/program (i.e., perceived as an adversary) and, thus, may have more difficulty obtaining relevant information.
- Stake in the evaluation
- Internal evaluator(s): May not be seen as an honest broker but may be perceived as having an agenda / stake in the evaluation.
- External evaluator(s): Can serve more easily as an arbitrator or facilitator between stakeholders as perceived as neutral.
- Internal evaluator(s): May be perceived as biased when perceived as ‘too close’ to the subject matter which may result in lesser credibility of the evaluation hindering its use.
- External evaluator(s): May provide a view of the project/program that is considered more objective, not part of the organization’s power structure, and thus, give the findings more credibility and potential for use.
- Internal evaluator(s): May use considerable staff time which is always in limited supply, especially when their time is not solely dedicated to the evaluation.
- External evaluator(s): May be more costly and still involve substantial management time from the commissioning organization’s staff.
- Follow-up / Use of evaluation findings
- Internal evaluator(s): More opportunity and authority to follow up on recommendations of the evaluation.
- External evaluator(s): Contracts often end with the delivery of the final product, typically the final evaluation report which limits or prohibits follow-up. As outsiders, do not have authority to require appropriate follow-up or action.
It is advisable to engage an external evaluator / evaluation team when:
- The scope and/or complexity of the evaluation demand expertise that is not internally available;
- A program or project is politically sensitive and impartiality is a key concern; or,
- Internal staff resources are scarce and timeframes are particularly pressing (i.e., there is little flexibility in terms of evaluation timing).
External evaluators may be an individual, a research institute or a consulting firm.
Planning and executing an evaluation is, in any case, a team effort. A critical decision to be made at this stage is who will lead the evaluation. Consider the following options for assembling the evaluation team:
- External evaluator(s) –one of them serving as the team leader– supported by program staff
- Internal evaluator(s) –one of them serving as the team leader– supported by program staff
- An internal evaluator –serving as the team leader– supported by other internal evaluators and program staff but also external evaluator(s)
Even if an external evaluator is hired to conduct the evaluation, the program manager and other staff must be involved in the evaluation process. Staff are not only primary users of the evaluation but also participants in data collection (such as providing access to records, educating the evaluator about the project/program or being interviewed as a key informant) and/or other evaluation-related tasks. Be realistic about the amount of time needed for this involvement so staff schedules do not get over-burdened.
Although hiring an external evaluator may seem costly, it may, ultimately, be less expensive than channeling considerable staff time into the evaluation. A careful analysis of staff time costs compared to external consultant costs is needed before making a decision.
For partnership evaluations (i.e., co-funded by more than one organization), double-check whether any of the co-funders have a requirement for using a ‘third party’ evaluator (i.e., someone who is not affiliated, in any way, with any of the organizations involved) (see also Step 1).
IDRC uses internal (i.e., in-house staff) or external evaluators or a mix of both. See: Selecting an Evaluation Consultant or Team. Evaluation Guideline, February 2012.
IDRC external reviews between 2010 and 2015 included both a self-assessment and external review component.
'Decide whether the evaluation will be done by an external team, an internal team or a hybrid of both' is referenced in:
- Communication for Development (C4D) :