Pathways to professionalisation - Part 1: Professionalisation within the context of the AES

Patricia Rogers and Greet Peersman's picture 25th July 2018 by Patricia Rogers and Greet Peersman

Should evaluation aim to become a profession?  And if so, in what ways?  Should evaluation become professional like medicine or law, where only certified practitioners with qualifications from accredited courses can call themselves evaluators?  Or professional like management, where many people are expected to use management skills, even if they are not called ’managers’ and don’t necessarily have formal qualifications?  Or is there a middle ground that might be more appropriate?

This debate has been going on for many years.  It has often arisen in response to instances of poor quality or unethical evaluations which have brought evaluation into disrepute and caused harm.  The discussion has gathered additional impetus more recently as other disciplines, professions and occupations have been increasingly engaged in evaluative work – including economists undertaking not only cost-benefit analyses but other types of evaluations, and the rise of impact investing, social impact analysis, market research, and data analytics.

As with any intervention, choosing the appropriate pathway depends on what you want to achieve.  If the primary aim is to protect the employment opportunities of existing evaluators, then a system of compulsory credentialing, with ‘grandfathering’ clauses which automatically include those currently practising, would be most appropriate.  If the primary aim is, instead, to improve the quality of evaluation, then it would be most appropriate to focus on approaches to support ongoing capacity strengthening of evaluators and others are engaged in evaluation. 

Four different pathways

We were delighted to be engaged by the Australasian Evaluation Society (AES) to explore options within the context of one of their strategic objectives ‘to strengthen the capacity and professionalism of the evaluation sector’. We undertook a literature review of the extensive debate about professionalisation in evaluation, talked with some of the people involved in initiatives internationally, explored how some associations in similar areas have addressed this started and developed an astonishing list of 41 different strategies that can and have been used for professionalisation.

Our report, which is available here, sets out four possible pathways to professionalisation for the AES;

Pathway 1

 Pathway 2

Pathway 3

Pathway 4

Business as usual:

Ad hoc, disconnected activities

Focused, connected and strategic activities

Voluntary credentialing of evaluators

Regulated and licensed profession

Pathway 1 is the current situation for the AES and most evaluation associations – they are engaged in a number of activities to improve the standard of practice, but these are neither connected, focused or strategic as activities in pathway 2 would be. Pathway 3 involves a voluntary credentialing process for individual evaluators and is being implemented in Canada and in Japan (for educational evaluators), while pathway 4 leads to a formal, regulated and licensed profession.

Our recommendation to the AES was not to pursue pathway 4, a regulated and licensed profession.  We didn’t see this as feasible or desirable given the diverse competencies needed to do different types of evaluation in different situations, the high investment needed, and the potential negative impacts of excluding competent practitioners for whom evaluation is not their primary identity.,

While pathway 3, voluntary credentialing of evaluators, might be feasible and useful in the future, we recommended that the AES should only proceed with this pathway after more consultation and evidence, including learning from the Canadian and Japanese experiences which have not yet been fully evaluated.

However, we did recommend a move away from pathway 1 to a more strategic approach, pathway 2, strengthening what the AES is currently doing through a purposeful focusing and connecting of its diverse activities. We also recommended additional activities. Some of these can be started right away, while others will need longer-term preparation and implementation. Some involve the AES acting alone and some will benefit from partnerships with other organisations, or are about influencing or supporting others to act.

In particular, we suggested that the AES:

  • Promote use of the AES Evaluators’ Professional Learning Competency Framework and Guidelines on Ethical Conduct of Evaluation and Code of Ethics.
  • Plan, develop and promote connected, ongoing professional development explicitly linked to identified priorities and the Evaluators’ Professional Learning Competency Framework, going beyond simply providing one-off training.
  • More systematically support sharing and learning from practice.
  • Become a more visible and effective advocate for evaluation and seek to influence demand and its enabling environment.
  • Engage in strategic partnerships with other evaluation associations and relevant local professional associations.

We also recommended that some level of wider consultation will be undertaken with AES members, other evaluators, and users of evaluation services to identify needs, resources, risks, opportunities and interest in engaging in various options. 

Figure 1. Pathway 2: START focusing, connecting and augmenting current activities

(View larger image in new window)

Our report is only the starting point or the initial input to a consultative and collaborative process led by the AES leadership and committees.  The AES’ board has accepted the report’s recommendations and the AES has already started mapping a way forward for pathway 2. The first stage of this includes consultation, and the AES will host a webinar in August for AES members to give input on this strategy (details to be announced).

For a more detailed look at our mapping of options for professionalisation in evaluation, see part two of this blog series.

Read more

Blog

Pathways to professionalisation - Part 2: Options for professionalisation

A major feature of the ‘Pathways to advance professionalisation within the context of the AES’ project and report is the exploration of 41 activities and approaches that can be used to advance the professionalisation of monitoring and evaluation. The conclusion of this two-part series looks at these approaches in more detail. We believe these activities are likely to be of considerable interest to others who are undertaking or planning evaluation capacity strengthening activities and we encourage you to share your feedback and thoughts on these activities at the end of this blog. Read more

Report

This report by Greet Peersman and Patricia Rogers for the Australasian Evaluation Society (AES) identifies four potential pathways towards professionalisation within the context of the AES: 1) Ad hoc, disconnected activities; 2) Focused, connected and strategic activities; 3) Voluntary credentialing of evaluators; and 4) Regulated and licensed profession, and identifies 41 activities and approaches that can be used to advance professionalisation. Read more
A special thanks to this page's contributors
Author
Deputy Director/BetterEvaluation Project & Associate Professor/ANZSOG, The Australia and New Zealand School of Government (ANZSOG).
Melbourne, Australia.
Author
Director of BetterEvaluation/ Professor of Public Sector Evaluation, Australia and New Zealand School of Government.
Melbourne.

Comments

Anonymous's picture
Lewis Atkinson

Hi there - I see a lot of similarities and parallels with how the Australasian Research Management Society dealt with beginning a transition to professional accreditation 10 years ago back in 2008.  However, they did also have an international model to follow as well.  See link to where they are now...in hindsight - despite the angst at the time - they think that they maade the right choices - https://researchmanagement.org.au/content/professional-development/ 

I was involved in the PD committee at the time.  Lewe

Patricia Rogers's picture
Patricia Rogers

Thanks for sharing this example.  For some reason the URL didn't work but I found the page at https://researchmanagement.org.au/content/professional-development/ (here)

It's very helpful to see how other associations have addressed this issue.  I see that in addition to accreditation, the Australasian Research Management Association also has a requirement for continuing professional education (CPE).  

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.