Whether you are an evaluator or someone commissioning evaluation, any intervention to be evaluated that takes place within human society and involves human interactions will have gendered dimensions. And that means that you as an evaluator should be able to identify and analyse those gendered dimensions.
But the way in which this analysis is done will depend on how the evaluator (and the intervention being evaluated) thinks about gender in the first place.
Note: Gender is one of many ‘markers of difference’ along which humans tend to judge each other. Other such markers include disability, ethnicity, gender identity, race, sexual orientation, sexuality, or socio-economic status. The ways in which these markers of difference intersect to increase marginalisation, exclusion and inequity is also vitally important for evaluators to keep in mind.
What do we mean by gender?
‘Gender affects everyone, all of the time. Gender affects the way we see each other, the way we interact, the institutions we create, the ways in which those institutions operate, and who benefits or suffers as a result of this.’ (Fletcher, 2015: Addressing Gender In Impact Evaluation: What Should Be Considered?)
The importance of considering gender is widely acknowledged in evaluation Terms of Reference, training curricula, evaluation-related publications and evaluation reports. But these documents often fail to clearly define what they mean by the term. There can be an assumption that the word is clearly understood, but in reality there is no one accepted way to understand what gender is. And that means there is no one accepted way of doing ‘gender analysis’. Similarly, there is no one way of doing feminist evaluation (as noted on the BetterEvaluation feminist evaluation theme page). This is about your way of thinking.
With that in mind, this theme page will begin by exploring the meaning of gender before moving on to issues of gender analysis.
There are many different definitions of gender, but the majority focus on unfair differences in the ways that women and men (categories of people) are treated in our societies. For example:
‘Gender refers to the roles, behaviours, activities, and attributes that a given society at a given time considers appropriate for women and men … In most societies there are differences and inequalities between women and men in responsibilities assigned, activities undertaken, access to and control over resources, as well as decision-making opportunities.’
Alternatively gender can be defined as:
‘a process of judgement and value … related to stereotypes and norms of what it is to be masculine or feminine, regardless of your born sex category … certain forms of femininity and masculinity are given greater value than others (with particular forms of dominant masculinity usually having the greatest access to power and resources).’
(Fletcher, 2015: Addressing Gender In Impact Evaluation: What Should Be Considered?)
What do we mean by gender analysis?
If one takes the first definition provided above as a starting point, then analysing the gendered aspect of an intervention would involve focusing on the differences between men and women within that intervention. For example, are there equal numbers of men and women involved in the intervention and, if not, why not.
If one takes the second definition as a starting point, then analysing the gendered aspect of an intervention would involve examining the judgements, stereotypes and norms related to masculinity and femininity that occur in the intervention’s context and, from there, exploring the effect these stereotypes and norms had on the intended intervention outcomes.
The difference between these two definitions is a crucial one when it comes to evaluating change. The first definition is looking for change in ‘differences and inequalities between women and men’ while the second one is looking for change in the ‘process of judgement and value’ that rewards certain forms of masculinity and femininity while punishing others.
This is an important difference. Gender analysis that is based on the seemingly binary categories of women/men looks for changes in the numbers, and ways, that men and women are engaged in an intervention.
Gender analysis that is based on processes of judgment, norms and stereotypes looks for whether, and how, changes have occurred during the intervention in judgements, stereotypes and norms related to masculinity and femininity, and the effects of those changes.
Taken from a Guidance Note on impact evaluation (Fletcher, 2015: Addressing Gender In Impact Evaluation: What Should Be Considered?), the following table demonstrates the gendered dimensions of an intervention, when analysed for its effect on (a)gender as process of judgement, and (b) gender as category (men/women). This work draws on Patricia Rogers’ (2012) key evaluation questions for impact evaluation:
Table 1: Gender as a process vs. gender as a category in a road-building intervention
Gender as process
Gender as category
Intervention's understanding of gender
Gender understood as a process that affects everyone
Gender understood in categorical terms: women = disadvantaged; men = advantaged
Increased access to services and markets for goods
For whom, in what ways and in what circumstances
The intervention would be considered to have been successful in relation to gender if:
Communities affected by the project consider and find ways to respond to questions of power and control, identifying and responding to inequalities, such as in:
N.B. While the strong likelihood is that the poorest women in a community will have the least access and least benefit from the road, adopting this approach ensures that the inequities existing within a community can be identified and questioned by those involved. The focus shifts from women, per se, to the power inequities at play.
The intervention would be considered to have been successful in relation to gender if:
N.B. While achievement of the above would be highly valuable, it would be unlikely to occur without the underlying causes being addressed. A categorical approach focuses on groups of people (i.e. women and men) and not on underlying gender processes.
Did the impacts match the needs of the intended beneficiaries?
This depends on the intervention's ability to engage the support and active participation of community members (including those with, and without, community sanctioned decision-making power) in identifying and questioning the stereotypes, norms and structures that underlie power inequities within those communities.
Important issues and considerations
It is worth repeating that every evaluation of an intervention that occurs within human society and involves human interactions will have gendered dimensions, whether or not these gendered dimensions are acknowledged within the intervention design, explicit or implicit theory of change (an implicit theory of change can also be understood as a ‘mental model’) and/or intervention implementation.
If, having examined an intervention’s documentation and having spoken to key stakeholders you can find no reference at all to gender in any form (gender as man/woman or gender as process of judgements, stereotypes and norms on masculinity and femininity), it is fair to identify the intervention as ‘gender blind’. In other words, it ‘ignores gender considerations all together’ (Interagency Gender Working Group, undated).
The implications of this would then need to be explored; and the depth of the analysis undertaken in exploring this will depend in large part on the resources available for the evaluation. It may well be that your gender analysis of such an intervention cannot go much beyond saying ‘this intervention is gender blind, and as such falls below the minimum standards expected’.
An intervention that ‘examines and addresses … gender considerations’ can be defined as ‘gender aware’ (Interagency Gender Working Group, undated) but then needs to be further analysed to see where it lies on what has been called ‘the gender continuum’:
An ‘exploitative’ intervention with regards to HIV prevention would be one that takes cultural judgements around gender and sexuality then uses them to promote ‘good’ behaviour: telling young women they will be ‘spoilt’ if they have sex before marriage, or saying that men who have sex with men (and who are often judged within society as ‘not real men’) are vectors of disease, when the risk lies in unprotected anal sex regardless of the sex of the bodies involved.
An ‘accommodating’ intervention would be one that ‘does not rock the boat’, for instance by claiming it would be ‘culturally inappropriate’ to involve sex workers or trans women in ‘women’s projects’ or accepting that, in an intervention working with civil society organisations, all those organisations are run by men (who will no doubt be from the majority religious and ethnic group in the intervention site).
A ‘transformative’ intervention would be one that works on gender processes; in other words, one that goes beyond the men/women categorical approach and that looks at judgements, stereotypes and norms of masculinity and femininity, and how they are applied regardless of the sex assigned to a body at birth. Referring back to Addressing Gender in Impact Evaluation:
Focusing on the stereotypes, norms and judgements related to masculinity and femininity (rather than on male/female) frees us up to think about the processes through which certain forms of femininity and masculinity are given greater value than others (with particular forms of dominant masculinity usually having the greatest access to power and resources).
Examples would include an intervention that shifted people’s ideas of what it takes to be ‘a leader’ (often considered to be something that requires certain traits seen as being dominantly masculine, such as lack of compassion and willingness to engage in conflict over conciliation; take for example the former UK Prime Minister Margaret Thatcher). An intervention that supported women’s right to contraception could also be gender transformative, if it did so by challenging the view (often held by women, as well as by men) that women who do not have children are somehow ‘lesser’ or that sex for pleasure is incompatible with ‘femininity’.
It is important to know which category the intervention to be evaluated falls into because this will determine, in part, what you can expect to achieve with using a gender analysis as part of your evaluation. In all instances, however, the role of the evaluator is to bring to light the positive and negative effects of the intervention in relation to gender. Sometimes these effects will be intended; often they will be unintended.
How to do a gender analysis
There are many different tools promoted for undertaking gender analysis. However many of these are based on a purely categorical understanding of gender, where ‘men’ and ‘women’ are seen as homogenous groups that are somehow in opposition to each other. The dynamics of gender—the ways in which we all participate in (re)creating norms and stereotypes against which we judge ourselves and others—are usually lost.
The reality is that there is no single, sure fire, A-Z way of doing gender analysis. But there are key steps to follow:
Question yourself: How do you think about gender? (How aware are you of the intersections between gender and other markers of difference that feed inequity?
Identify whether the intervention to be evaluated is gender blind or gender aware: Start with a desk review of key documents, such as funding applications/design documents; monitoring reports; theory of change or other logic model if available; but supplement this using different methods: see following section. N.B.: doing a word search on ‘gender’ in key documents is not the same as analysing the documents for gendered content. It is entirely possible for an intervention to not actually use the word ‘gender’ but to still challenge stereotypes and norms. Equally, there are many interventions where the documents use the word ‘gender’ widely but the intervention itself is gender blind.
Gender blind interventions: If, based on this first stage, the intervention is gender blind (in that it pays no attention to gender, either in categorical terms or in terms of processes of judgement), then say so! If resources allow, it would also be appropriate to draw out some of the consequences of this gender blindness. There are many resources to draw on that clearly define gender as a required component of evaluation of interventions that take place within human society and involve human interactions. (This page is one such resource; others include the Australian Department of Foreign Affairs and Trade Monitoring and Evaluation Guidelines, the Australasian Evaluation Society Guidelines for the Ethical Conduct of Evaluations, which deserves praise for also requiring account to be taken of other markers of difference, not just gender; and the United Nations Evaluation Group, whose latest resource is listed below.)
Gender aware interventions: If, based on stage two above, you determine that the intervention is gender aware, how would you describe its understanding of gender? There may not be an explicit explanation—this is often missing—but you will be able to tell by what is done, with whom, and how. Refer back to Tables 1 and 2 for examples of how the different types of gender might play out in the same intervention. Easy questions to keep in mind are: ‘does this intervention engage differently with men and women?’ or ‘does this intervention engage with judgements, stereotypes and norms about masculinity and femininity?’. As we are talking about evaluation, your over-arching questions for either understanding of gender should be: What gendered changes have occurred as the intended, or unintended, results of the intervention? (descriptive question) What explains those changes? (causal question) and, if you are undertaking an impact evaluation, ‘What does this say about the value of the intervention?’ (evaluative question) . If the evaluation is a process evaluation, then you would need to think about ‘How can the intervention be adapted to reduce any negative changes, increase positive changes, and ensure that the intervention’s theory of change or mental model is appropriate?
There is a large range of methods available to the evaluator as they move through the steps above. As with all forms of social enquiry, the method adopted should be coherent with your evaluation logic. What do you want to know, who can help you find that out, and finally which method(s) are most appropriate? Selection of method cannot be your starting point in gender analysis; thinking your logic through has to come first.
And remember; as with feminist evaluation, gender analysis is, after all, an exercise in exploring inequities and promoting social justice. As such it sits most comfortably with ‘participatory, empowering’ methods (Podems, 2016).
The Institute of Development Studies manages a Participatory Methods website that has a specific section on participatory monitoring and evaluation, as well as providing details on a wide range of methods that can be used to support gender analysis.
The World Bank Group also has a list of methods ranging from desk review to stakeholder workshops and social mapping, available here.
Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized Voices (ISE4GEMs): A New Approach for the SDG Era: This guidance paper by Anne Stephens, Ellen D. Lewis , and Shravanti Reddy aims to bring together complexity, systems thinking and boundary analysis (first order, second order), intersectionality, the environment, gender equality and ‘marginalized voices’ into a coherent whole, systems thinking is not the only way to approach evaluation of complexity. Aimed at experienced evaluators, what this paper does well is to consistently acknowledge that ‘women’ are not a homogeneous group, as well as supporting use of an intersectional approach. It also challenges evaluators to be self-reflexive. But to use the document’s own definition, it is ‘chaotic … only turbulence exists’. The document has also been criticised for making just five references to disability.
Addressing Gender In Impact Evaluation: What Should Be Considered?: This guide by Gillian Fletcher helps to clarify the meaning of gender as socially constructed norms and expectations around masculinity and femininity as opposed to the common misunderstanding of gender as the biological sex difference between men and women. The guide explains the implications of this crucial distinction for categorizing interventions with an explicit or implicit gender focus and for assessing their impact on gender-related injustice and inequality. The guide recommends specific steps in assessing gender-related impact including: understanding and classifying how an intervention seeks to engage with gender; defining gender-sensitive evaluation questions to address gender impact; and, methods and tools that are particularly helpful in answering such questions.
Gender-Based Analysis Plus (GBA+): Developed by Status of Women Canada (2017), GBA+ is an analytical process used to assess how diverse groups of women, men and non-binary people may experience policies, programs and initiatives. The “plus” in GBA+ acknowledges that GBA goes beyond biological (sex) and socio-cultural (gender) differences. We all have multiple identity factors that intersect to make us who we are; GBA+ also considers many other identity factors, like race, ethnicity, religion, age, and mental or physical disability.’ The Government of Canada is committed to using GBA+ analysis in all of its work. The link provided above enables access to a range of materials and resources, including a ‘GBA+ Research Guide’. While still taking a firmly categorical approach that prioritises types of individuals over process of exclusion, GBA+ is a very interesting example of a way in which government can shape these debates.
Engendering Transformative Change in International Development: Engendering Transformative Thinking and Practice in International Development draws on a range of real world examples which demonstrate both the limitations of the frameworks currently in use, and the very real possibilities for change when the intersecting social hierarchies that sustain and create inequity and inequality are challenged. This book brings together theoretical perspectives on social change, gender, intersectionality, and forms of knowledge, concluding with a set of proposals for revitalising a change agenda that recognises and engages with intersectionality and practical wisdom. Perfect for students and scholars of social change, gender, and development, this book will also be useful for practitioners looking for new ideas to help to generate social change.
Using the Social Relations Approach to capture complexity in women's empowerment: using gender analysis in the Fish on Farms project in Cambodia: This article by Emily Hillenbrand et al. (2014) discusses the application of Naila Kabeer’s Social Relations Approach (SRA) to gender analysis ‘to frame a baseline gender analysis of a food security project undertaken in Cambodia’. The project was being implemented as a randomised control trial; this paper demonstrates the ways in which such an approach is not ‘fit for purpose’ when trying to shift social norms, because it does not allow for adaptation and change based on learning generated during implementation. However use of the SRA helped those involved to highlight ‘the nuances and complexities of gender relations’, as well as ‘listening closely – and responding to – what women (and men) themselves consider to be empowering and positive indicators of change’. The authors noted: ‘An analysis approach such as this that emphasises processes of gender change and the dynamic nature of gender relations and institutional changes also requires effective process-monitoring and learning tools that facilitate introspection and interpretation of changes over time’ (emphasis in original).
Tackling Antimicrobial Resistance (AMR) Together. Working Paper 5.0: Enhancing the Focus on Gender and Equity: The aim of this WHO working paper is to assist countries to better consider issues of gender and equity in antimicrobial resistance (AMR) efforts, and to 'inform the implementation of strategies in national action plans and contribute to improved reach and effectiveness of AMR efforts in the longer term.’ While highly health specific, the Working Paper offers a range of interesting and insightful reflections on gender analysis, for example: ‘A gender analysis can be used to review health education and behaviour change strategies to check that they are not gender blind (i.e. ignoring the differences in opportunities and resource allocation for women and men) and or don’t reinforce gender inequalities by relying on traditional gender norms and stereotypes e.g. assuming that young men are not interested in healthy behaviours and are not amenable to behaviour change’.
Fletcher, G. (2015). Addressing gender in impact evaluation. A Methods Lab Publication. London: Overseas Development Institute & Melbourne: BetterEvaluation. Retrieved from www.betterevaluation.org/en/resources/addressing_gender_in_impact_evalua...
Fletcher, G. (2018). Engendering transformative change in international development. London, New York: Routledge. 186pp.
Hillenbrand, E. et al. (2014) 'Using the Social Relations Approach to capture complexity in women's empowerment: using gender analysis in the Fish on Farms project in Cambodia'. In Gender & Development, 22:2, pp.351-368. Retrieved from: www.tandfonline.com/doi/full/10.1080/13552074.2014.920992
Interagency Gender Working Group (n.d.). Gender Integration Continuum. Retrieved from: www.igwg.org/wp-content/uploads/2017/05/FG_GendrIntegrContinuum.pdf
Podems, D. and Negroustoueva, S. (2016) Feminist evaluation. BetterEvaluation. Retrieved from http://www.betterevaluation.org/approaches/feminist_evaluation
Status of Women Canada (2017). Gender-Based Analysis Plus (GBA+). Ottawa, Canada: Status of Women Canada. Retrieved from: cfc-swc.gc.ca/gba-acs/index-en.html
Stephens, A., Lewis, E.D. and Reddy,S.M. (2018). Inclusive Systemic Evaluation (ISE4GEMs): A New Approach for the SDG Era. New York: UN Women. Retrieved from http://www.unwomen.org/en/digital-library/publications/2018/9/ise4gems-a...
WHO (2018). Tackling Antimicrobial Resistance (AMR) Together. Working Paper 5.0: Enhancing the Focus on Gender and Equity. Geneva: World Health Organization. Retrieved from: www.who.int/antimicrobial-resistance/national-action-plans/AMRGenderEqui...