C4D: Specify the key R,M&E questions

What is it?

R,M&E Questions are the small number of broad questions that R,M&E are intended to answer, not the many specific questions that might be on a questionnaire or an interview schedule. Deciding which questions should be answered is one of the most important and often the most difficult parts of designing M&E (Catley et al. 2008: 12). The approach advocated here is a questions-driven approach, where key users first agree on what they need to know and use that as the basis for selecting methods and indicators. The evaluation part of R,M&E systems, by definition, should answer truly evaluative questions: it must ask not only ‘What were the results?’ (a descriptive question) but also ‘How good were the results?’ (an evaluative question). Depending on the type of M&E, causal questions also need to be addressed (to what extent were the results due to the intervention?).

General Information 

The approach advocated on this page draws heavily from the following pages in the Manager's Guide to Evaluation:

• Develop agreed key evaluation questions

  Evaluation, by definition, must answer truly evaluative questions: it must ask not only ‘What were the results?’ (a descriptive question) but also ‘How good were the results?’ (an evaluative question). Depending on the type of evaluation, causal questions also need to be addressed (to what extent were the results due to the intervention?).
• Consider important aspects of the evaluation

  Evaluations are designed to answer the Key Evaluation Questions. Different types of questions need different methods and designs to answer them.

This particular resource brings clarity to the task of articulating questions and understanding the type of question being asked (descriptive, causal, evaluative and action), and therefore the kinds of methods that can be used to answer them. Other key generalist advice includes:

• Limit the number to 5-7 high level questions
• Understand the kinds of questions asked at different points in the program cycle
• Use the purpose to guide the selection of questions 

The Specify the Key Evaluation Questions page of the Rainbow Framework similarly offers generalist advice, and presents the task in a slightly different way. It includes some good links to guide on engaging with stakeholders to develop evaluation questions, which is useful for taking a participatory approach to specifying questions. These pages are recommended background reading before considering options to apply to C4D. 

Why it is useful to analyse the types of questions within key questions for C4D

Observations as part of the Evaluating C4D project with UNICEF have revealed two problems:

• People often think that indicators come first, and questions are developed based on these
• Relatedly, R,M&E work tends to mostly focus on describing situations, and it is assumed that from there it is easy to infer contribution and causation, and decide on actions.

In particular, questions about contribution and attribution in C4D are common themes in discussions about needs, but causal questions and methods are rarely incorporated in C4D R,M&E designs. If questions about C4D contributions are important for stakeholders, it is vital that causal questions are reflected in the key questions (most likely as sub-questions) selected.

When deciding on key questions for C4D R,M&E the following steps are recommended:

1. Use the Program Theory or Logic Model  
2. Consider the types of key questions
3. Analyse each Key Question  

Applying the C4D principles

Participatory

The C4D Evaluation Framework advocates for a participatory approach. In the context of specifying key questions, a participatory approach would mean engaging (at least) with primary intended users and other stakeholders to decide on key questions.

Holistic

C4D initiatives usually respond to problems strongly connected with different social, cultural, economic, political, geographic and structural contexts. This means that in C4D R,M&E it is important to ask questions about underlying causes and social, cultural economic, political, geographic and structural contexts - from the situation analysis right through to the monitoring and evaluation.

Critical

In C4D it is important that questions are framed in such a way that allows for multiple and diverse voices to contribute answers. This is important for descriptive questions, causal questions and evaluative questions.

Realistic

In C4D the questions should be written in a way that calls for need for various methods and tools that will capture people's voices.

Complex

It is likely that there will be differing views that need to be taken into account about what the key R,M&E questions should be. In addition, the boundaries may need adjusting as situations change, particularly with the emergence of new understandings, stakeholders and ideas.