Pathways to professionalisation - Part 1: Professionalisation within the context of the AES
In part 1 of this two-part blog series, greet Peersman and Patricia Rogers introduce the ‘Pathways to advance professionalisation within the context of the AES’ project and report.
They explore the four pathways identified in the report: 1) Ad hoc, disconnected activities; 2) Focused, connected and strategic activities; 3) Voluntary credentialing of evaluators; and 4) Regulated and licensed profession, and discuss their recommendation that the Australasian Evaluation Society follow a pathway of focused, connected and strategic activities, with a view to considering a voluntary credentialing process down the track.
Should evaluation aim to become a profession? And if so, in what ways? Should evaluation become professional like medicine or law, where only certified practitioners with qualifications from accredited courses can call themselves evaluators? Or professional like management, where many people are expected to use management skills, even if they are not called ’managers’ and don’t necessarily have formal qualifications? Or is there a middle ground that might be more appropriate?
This debate has been going on for many years. It has often arisen in response to instances of poor quality or unethical evaluations which have brought evaluation into disrepute and caused harm. The discussion has gathered additional impetus more recently as other disciplines, professions and occupations have been increasingly engaged in evaluative work – including economists undertaking not only cost-benefit analyses but other types of evaluations, and the rise of impact investing, social impact analysis, market research, and data analytics.
As with any intervention, choosing the appropriate pathway depends on what you want to achieve. If the primary aim is to protect the employment opportunities of existing evaluators, then a system of compulsory credentialing, with ‘grandfathering’ clauses which automatically include those currently practising, would be most appropriate. If the primary aim is, instead, to improve the quality of evaluation, then it would be most appropriate to focus on approaches to support ongoing capacity strengthening of evaluators and others are engaged in evaluation.
Four different pathways
We were delighted to be engaged by the Australasian Evaluation Society (AES) to explore options within the context of one of their strategic objectives ‘to strengthen the capacity and professionalism of the evaluation sector’. We undertook a literature review of the extensive debate about professionalisation in evaluation, talked with some of the people involved in initiatives internationally, explored how some associations in similar areas have addressed this started and developed an astonishing list of 41 different strategies that can and have been used for professionalisation.
Our report sets out four possible pathways to professionalisation for the AES:
Pathway 1 (Business as usual)
Ad hoc, disconnected activities
Focused, connected and strategic activities
Voluntary credentialing of evaluators
Regulated and licensed profession
Pathway 1 is the current situation for the AES and most evaluation associations – they are engaged in a number of activities to improve the standard of practice, but these are neither connected, focused or strategic as activities in pathway 2 would be. Pathway 3 involves a voluntary credentialing process for individual evaluators and is being implemented in Canada and in Japan (for educational evaluators), while pathway 4 leads to a formal, regulated and licensed profession.
Our recommendation to the AES was not to pursue pathway 4, a regulated and licensed profession. We didn’t see this as feasible or desirable given the diverse competencies needed to do different types of evaluation in different situations, the high investment needed, and the potential negative impacts of excluding competent practitioners for whom evaluation is not their primary identity.,
While pathway 3, voluntary credentialing of evaluators, might be feasible and useful in the future, we recommended that the AES should only proceed with this pathway after more consultation and evidence, including learning from the Canadian and Japanese experiences which have not yet been fully evaluated.
However, we did recommend a move away from pathway 1 to a more strategic approach, pathway 2, strengthening what the AES is currently doing through a purposeful focusing and connecting of its diverse activities. We also recommended additional activities. Some of these can be started right away, while others will need longer-term preparation and implementation. Some involve the AES acting alone and some will benefit from partnerships with other organisations, or are about influencing or supporting others to act.
In particular, we suggested that the AES:
- Promote use of the AES Evaluators’ Professional Learning Competency Framework and Guidelines on Ethical Conduct of Evaluation and Code of Ethics.
- Plan, develop and promote connected, ongoing professional development explicitly linked to identified priorities and the Evaluators’ Professional Learning Competency Framework, going beyond simply providing one-off training.
- More systematically support sharing and learning from practice.
- Become a more visible and effective advocate for evaluation and seek to influence demand and its enabling environment.
- Engage in strategic partnerships with other evaluation associations and relevant local professional associations.
We also recommended that some level of wider consultation will be undertaken with AES members, other evaluators, and users of evaluation services to identify needs, resources, risks, opportunities and interest in engaging in various options.
Figure 1. Pathway 2: START focusing, connecting and augmenting current activities
Our report, Pathways to Advance Professionalissation within the context of the AES, is only the starting point or the initial input to a consultative and collaborative process led by the AES leadership and committees. The AES’ board has accepted the report’s recommendations and the AES has already started mapping a way forward for pathway 2. The first stage of this includes consultation, and the AES will host a webinar in August for AES members to give input on this strategy (details to be announced).
For a more detailed look at our mapping of methods for professionalisation in evaluation, see part two of this blog series.