Reflections on meeting the challenge of communicating the validity of culturally responsive evaluation (CRE) and getting influential voices and changemakers to listen

The following article was written by Ennie Flora Gatsi (MA), Rumbidza Tizora (MSc), and Joyce Muyengwa (MSc). Congratulations on your winning entry to the Young and Emerging Evaluators (YEEs) essay competition! The competition, jointly hosted by the Independent Evaluation Group (IEG), the Global Evaluation Initiative (GEI), and EvalYouth invited emerging evaluators to propose solutions for closing the gap between global and local knowledge. 

Introduction

Culturally-Responsive Evaluation (CRE) is an approach to evaluation that acknowledges the subjectivity and diversity of knowledge, points of view, and ways of knowing that guide evaluations (EvalCommunity, n.d.). It stresses the impact of evaluators' knowledge and views on their ability to connect with, comprehend, and provide meaningful results that are relevant to the social and cultural contexts of the communities, cultures, and societies with which they work (Hood et al., 2015). Despite its significance, explaining the legitimacy of CRE to prominent voices and change-makers/decision-makers may be difficult.

Understanding culturally responsive evaluation

CRE is a technique for evaluation that recognizes the importance of culture in shaping people’s experiences and perspectives (Gervin et al., 2014). It acknowledges that culture influences how people perceive and interpret information, communicate, and interact with others. As a result, it emphasizes the importance of interacting with communities in a culturally sensitive manner to guarantee that evaluations are relevant, meaningful, and beneficial to the planned use (Hood et al., 2015). CRE draws on and extends the concepts of previous evaluation methodologies such as responsive evaluation, democratic evaluation, and participatory evaluation (Sanakar, 2017).

The importance of culturally responsive evaluation

CRE is important because it recognizes that evaluations are not neutral or objective but are shaped by evaluators’ knowledge, perspectives, and ways of knowing (EvalCommunity, n.d.). It recognizes that if evaluations are not done in a culturally sensitive manner, they may have unexpected repercussions. For example, evaluations that do not account for cultural variations may provide conclusions that are irrelevant or meaningless to the people they serve. They may also exacerbate existing power inequities by favouring some types of information over others.

Challenges in communicating the validity of culturally responsive evaluation

Despite its importance, communicating the validity of culturally responsive evaluation can be challenging. One of the challenges is that CRE is often seen as a niche approach that is only relevant to certain populations or contexts (Bryan & Lewis, 2019). This perception can make it difficult to convince influential voices and change-makers of its value.

Another challenge is that CRE requires a significant investment of resources, especially time. It involves engaging with communities in a culturally sensitive manner, which can be time-consuming and require specialized skills. This investment can be difficult to justify to funders or other stakeholders who may not understand the importance of CRE.

Finally, there is a lack of consensus on what constitutes CRE. This lack of clarity can make it difficult to communicate the validity of CRE to influential voices and change-makers who may have different understandings of what CRE entails.

Strategies for communicating the validity of culturally responsive evaluation

Despite these challenges, there are several strategies for communicating the validity of culturally responsive evaluation to influential voices and change-makers. These include:

1. Develop definitions

Developing clear definitions of what constitutes culturally responsive evaluation can help dispel misconceptions about its relevance and value. It can also help to ensure that stakeholders have a shared understanding of what culturally responsive evaluation entails, which can facilitate communication and collaboration. Clear definitions can also help to ensure that evaluations are conducted in a culturally sensitive manner, which can increase their relevance, meaning, and usefulness.

For example, clear definitions can help to distinguish between culturally responsive evaluation and other approaches to evaluation, such as multicultural evaluation or cross-cultural evaluation. This distinction can help to clarify the unique contributions of culturally responsive evaluation and its relevance to different contexts and populations.

In addition, clear definitions can help to ensure that evaluations are conducted in a culturally sensitive manner by providing guidance on how to engage with communities, how to collect data, and how to interpret the findings. This guidance can help evaluators avoid common pitfalls, such as imposing their own cultural assumptions on the evaluation process or failing to recognize the diversity of perspectives and experiences within a community.

Overall, developing clear definitions is an important strategy for communicating the validity of culturally responsive evaluation. It can help to dispel misconceptions about its relevance and value, ensure that stakeholders have a shared understanding of what it entails, and provide guidance on how to conduct evaluations in a culturally sensitive manner.

2. Engage with stakeholders

Engaging with stakeholders throughout the evaluation process can help to build trust and ensure that evaluations are relevant, meaningful, and useful (Frierson et al., 2002). It can also help to ensure that evaluations are conducted in a culturally sensitive manner by providing opportunities for community members to share their perspectives, experiences, and knowledge. This engagement can take many forms, such as community meetings, focus groups, or interviews.

Engaging with stakeholders can also help to build support for culturally responsive evaluation by providing opportunities for influential voices and change-makers to learn about its value and relevance (Hood et al., 2015). This engagement can help to dispel misconceptions about the approach and build momentum around its use.

Overall, engaging with stakeholders is an important strategy for communicating the validity of culturally responsive evaluation. It can help to build trust, ensure that evaluations are conducted in a culturally sensitive manner, and build support for the approach among influential voices and change-makers.

3. Provide evidence

Providing evidence of the effectiveness of culturally responsive evaluation can help to convince influential voices and change-makers of its value. Providing evidence involves collecting data that supports the validity of the evaluation. The data should be collected in a way that is sensitive to the cultural context of the program and participants (Frierson et al., 2002). The evaluator should also consider how to present the data in a way that is culturally appropriate and meaningful to the participants (Frierson et al., 2002).

In summary, providing evidence is an important strategy for communicating the validity of culturally responsive evaluation. It involves collecting data that supports the validity of the evaluation while being sensitive to the cultural context of the program and participants (Hood et al., 2015). The evaluator should also consider how to present the data in a way that is culturally appropriate and meaningful to the participants.

4. Build capacity

Building evaluators’ capacity to conduct culturally responsive evaluations can help to ensure that evaluations are conducted in a culturally sensitive manner.

Building capacity can take many forms, such as training sessions, workshops, and mentoring programs (Frierson et al., 2002). These activities should be designed to meet the specific needs of the stakeholders involved in the evaluation process. For example, training sessions could focus on developing skills in data collection or analysis, while mentoring programs could provide support for evaluators working in culturally diverse settings (Frierson et al., 2002).

In summary, building capacity is an important strategy for communicating the validity of culturally responsive evaluation. It involves developing the skills and knowledge of stakeholders involved in the evaluation process through training sessions, workshops, and mentoring programs. By building capacity, stakeholders can better understand the cultural context of the program and the importance of culturally responsive evaluation.

5. Collaborate

Collaborating with other evaluators or organizations can help to build momentum around culturally responsive evaluation and increase its visibility. To communicate the validity of CRE collaboration is a crucial strategy (Askew et al., 2012). Collaboration involves working with stakeholders to develop a shared understanding of the cultural contexts of the programs and the importance of CRE. This includes program staff, participants, and evaluators. By collaborating, stakeholders can work together to ensure that the evaluation is culturally appropriate and meaningful to the participants.

Collaboration can take many forms, such as focus groups, community meetings, and advisory committees (Innovation Collaborative, 2020.). These activities should be designed to meet the specific needs of the stakeholders involved in the evaluation process. For example, focus groups could be used to gather feedback from participants on the cultural relevance of the evaluation, while advisory committees could provide guidance on how to present data in a way that is culturally appropriate (Innovation Collaborative, 2020).

In summary, collaboration is an important strategy for communicating the validity of culturally responsive evaluation. It involves working with stakeholders to develop a shared understanding of the cultural context of the program and the importance of culturally responsive evaluation through focus groups, community meetings, and advisory committees. By collaborating, stakeholders can work together to ensure that the evaluation is culturally appropriate and meaningful to the participants.

Culturally responsive assessments should completely consider the culture of the programme being assessed. This indicates that the evaluation should be based on an investigation of impacts via lenses that consider the participants' culture to be a significant aspect, rejecting the concept that evaluations must be objective and culture-free. Evaluators who are culturally sensitive should respect the cultural context in which an evaluation is taking place by bringing required, shared life experience and understandings to the evaluation tasks.

Conclusion

In conclusion, culturally responsive evaluation is an important approach to evaluation that recognizes the importance of culture in shaping people’s experiences and perspectives. However, communicating its validity to influential voices and change-makers can be challenging. Strategies for addressing this challenge include developing clear definitions, engaging with stakeholders, providing evidence, building capacity, and collaborating with others.