Week 37: Why is equity so important in evaluation?
This is #3 in our series on visionary evaluation.
I first started to think about equity as an issue in my work as a ‘development practitioner’ as a realistic alternative to the calls by development donors such as the Australian Aid Agency (AusAID) for ‘gender equality’. I say a realistic alternative because in the area of community development, which is where my work is mostly located, gender equality is so difficult to achieve in terms of equal numbers of men and women participating and in terms of equal outcomes for men and women. There are many reasons for this but they can be summed up as ‘barriers to participation’ and these barriers are mostly gendered. For example, literacy requirements, time availability and timing of activities usually disadvantage women, whereas projects such as micro finance and savings and loans setups generally target women and can disenfranchise men. When you add to these realities, the facts that culture, ethnicity and religion in developing countries (and even for ethnic minorities in developed countries) achievement of equality, especially within the timeframe of a development project, is almost impossible. Equality, then, is more of a strategic objective whereas equity is something which can be achieved in the short term.
The UNICEF my M&E website describes equity as, “...being fair to both men and women”. Being fair has an aspect of consideration for the needs or different requirements of men and women. Equality usually means the same for everyone.
Equity also has an ethical aspect to it. Treating people equally or the same may, in some cases, be unethical. For example, using written surveys or questionnaires with persons who are illiterate or cannot read the language in which the document is written, would be an unethical practice. Using a method which everyone can understand is both equitable and ethical.
In relation to evaluation, achieving equity can be a little more difficult than achieving it in project implementation. In evaluation there are three areas in which equity should be the objective – the informant or data collection area, the findings and conclusions of the evaluation and, then, in the recommendations arising from the evaluation.
I used to think that if I ensured that all the barriers to participation were minimised through the use of user-friendly and culturally appropriate data collection tools then participation would be maximised, and thus, ensuring that both men and women would be able to participate (Donnelly, 2010). I now realise that this was based on two assumptions – that all men and women wanted to participate but some were prevented by the ‘barriers to participation’, and that everyone had something to contribute to the evaluation. Experience has shown me that these assumptions are not necessarily accurate. Not everyone wants to be involved and even if people do participate, if they do not have what the UNDP (2014) and Kumar (2002) refer to as, “...understanding of the internal dynamics of their project, its successes and failures…”, they may not have anything to contribute. In such cases their ‘ignorance’ could be seen as another barrier to participation – they have had no equity in the project/activity/intervention being evaluated. Informants then need to have had equity in the project/activity/intervention being evaluated and must be able to participate through the use of tools which are fair to all informants.
If people are unable to make an informed contribution to the evaluation, the lack of reliable data can influence the findings and the conclusions of the evaluation. The findings and conclusions, if drawn from uninformed/unreliable data may have a bias which may be unfair to either men or women or both. The recommendations drawn from such findings and conclusions may also be such that they potentially create an unfair situation for men or women (or both) in any future projects/activities/interventions derived from the evaluation outputs of findings, conclusions and recommendations.
To achieve equality in evaluations, it is important that the project/activity/intervention being evaluated was equitable. At the very least the informants to the evaluation must have had equity in the project/activity/intervention so that they can have an equitable role in the evaluation. It is also just as important that informants to the evaluation can contribute to the evaluation in a manner which is suitable to their abilities, customs and practices. It is far better to have quality/valid representation for men and women in an evaluation than to just ensure equal representation through numbers.
Equity in evaluation is enhanced further by involving men and women who wish to do so, in the consideration of the findings (analysis) to determine the conclusions (what was achieved) and the recommendations (how might we do it better next time). This level of involvement not only is equitable and ethical, it enhances equity in future projects/activities/interventions.
The real value of equity in evaluation is that through equity real equality in evaluation is achieved.
References:
Donnelly, J. (2010). Maximising participation in international community-level project evaluation; a strength based approach. Evaluation Journal of Australasia 10(2): 43-50.
Kumar, S. (2002). Methods for Community Participation: A complete guide for practitioners. London: ITDG Publishing.
UNDP. Participatory evaluation. http://web.undp.org/evaluation/documents/whop2.htm (accessed on 16 June 2014)
UNICEF. Equity-focused and gender responsive evaluations. https://ecourses.evalpartners.org/ecourses/course-details/1 (accessed on 16 June 2014)
Perspectives from others
Rosalind Eyben, Emeritus Fellow, Institute of Development Studies, University of Sussex, UK.
How does one define ‘equity in evaluation’? It is the opposite of ‘inequity’. Thus equity in evaluation requires rendering visible and taking into account the inequitable power relations that influence development programmes and projects.
If the programme being evaluated has explicit equity objectives, then I would want to do a power analysis to help me be alert to whose knowledge and evidence is likely to be overlooked unless I make an explicit effort to include them (and this can be politically risky if the local powers that be disapprove). And then I would need to use appropriate methodologies to enable the experience of the powerless to be included in the findings. Importantly, this may very well mean learning about the views and experience of people that have not been involved in the project. In the past, a classic error in evaluating primary health care or agricultural extension services, for example, was to interview only the service-users and not others equally eligible (at least in the eyes of the programme designers) who didn’t use them. Also, when the programme designers have made the wrong assumption about who would benefit from the service……
Equity in evaluation gets really interesting with respect to unintended consequences! How do you discover the causes of such consequences? My experience as a development practitioner has led me to think that (beneficial or otherwise) these are likely to be an effect of local social practices and the power relations associated with these. And, of course, there may be serious unintended inequitable outcomes from projects that were not designed with equity objectives. The project may be a success (dam built on time and urban populations getting more stable electricity supply) but local people in the project area find themselves homeless. Equity in evaluation means looking at the wider impact of a project not just whether it achieved its objectives.
Juliet Willetts, Institute for Sustainable Futures, University of Technology Sydney
Equity in evaluation, for me, could have many dimensions. But first it’s probably good to think about what ‘equity’ can mean to different people. For those working in the area of human rights, normally ‘equality’ is the formal term used rather than ‘equity’ since it has a legal basis. It is used in relation to ideas around non-discrimination, the legal principle that prohibits the less favorable treatment of individuals or groups based on aspects such as ethnicity, sex, religion, or other status.
Equity is also a term often used in relation to vulnerable or marginalised groups. These terms vulnerable and marginalised are often used interchangeably, yet they are different. Vulnerable refers to a trait or characteristic of a person which makes that person at risk to harm or injury (physical or emotional). Marginalised refers to those people who lack access to opportunities (for instance access to services) due to poverty, tenure status, lack of attention to remote areas or for reasons of discrimination. Both have equity implications. So what does this mean for evaluation. Many things.
Firstly, does the framing of evaluation questions allow for or promote exploration of equity issues that may have arisen in the context a policy or program? For instance is the distribution of benefits or outcomes across a given population given consideration in the framing of evaluation questions? If an evaluation doesn’t seek to reveal such differences in outcomes, they may well remain invisible. So a first point would be that there may need to be an explicit intent to ‘find out’ something about equity for it to be meaningful in the context of an evaluation.
Secondly, concerning equitable participation, John has raised some really important points: the choice to participate and not; capacity and suitability to participate; and suitable methods that facilitate meaningful participation of different groups.
These ideas raise some questions for me around evaluation ethics. Some work was recently undertaken in the aid and development sector by the peak body of the non-governmental organisations (ACFID) to develop principles and guidelines for ethics in research and evaluation in the context of aid and development. The principles are available. Soon to be released are additional guidelines, which include advice and links to resources on considering ethics when involving vulnerable groups in evaluation and research. These may be a helpful resource towards improving ‘equity’ in evaluation.
Resources
E-learning course on Equity-Focused and Gender-Responsive Evaluations
This e-Learning programme addresses the opportunities and challenges, as well as the methodological implications, of evaluating the effects of policies, programmes, and projects designed to enhance equitable development results. A special focus is directed towards effects on the most excluded, marginalized, and deprived groups. =
Related content
'Week 37: Why is equity so important in evaluation?' is referenced in:
Blog