Most Significant Change

Synonyms: 
MSC

The Most Significant Change (MSC) approach involves generating and analysing personal accounts of change and deciding which of these accounts is the most significant – and why.

The are three basic steps in using MSC:

  1. Deciding the types of stories that should be collected (stories about what - for example, about practice change or health outcomes or empowerment)
  2. Collecting the stories and determining which stories are the most significant
  3. Sharing the stories and discussion of values with stakeholders and contributors so that learning happens about what is valued.

MSC is not just about collecting and reporting stories but about having processes to learn from these stories – in particular, to learn about the similarities and differences in what different groups and individuals value.

It provides some information about impact and unintended impact but is primarily about clarifying the values held by different stakeholders. By itself it is not sufficient for impact evaluation as it does not provide information about the usual experience but about the extremes.  

If you imagine a normal distribution of outcomes for individuals then the stories often come from the extremity of positive change.  It can be useful to explicitly add a process to generate and collect stories  from the extremity of little or negative change.

MSC can be very helpful in explaining HOW change comes about (processes and causal mechanisms) and WHEN (in what situations and contexts). It can therefore be useful to support the development of programme theory (theory of change, logic models).

Example

"In 1994 Rick Davies was faced with the job of assessing the impact of an aid project on 16,500 people in the Rajshahi zone of western of Bangladesh (6). The idea of getting everyone to agree on a set of indicators was quickly dismissed as there was just too much diversity and conflicting views. Instead Rick devised an evaluation method which relied on people retelling their stories of significant change they had witnessed as a result of the project. Furthermore, the storytellers explained why they thought their story was significant.

If Rick had left it there the project would have had a nice collection of stories but the key stakeholders’ appreciation for the impact the project would have been minimal. Rick needed to engage the stakeholders, primarily the region’s decision-makers and the ultimate project funders, in a process that would help them see (and maybe even feel) the change. His solution was to get groups of people at different levels of the project’s hierarchy to select the stories which they thought was most significant and explain why they made that selection.

Each of the 4 project offices collected a number of stories and were asked to submit one story in each of the four areas of interest to the head office in Dhaka. The Dhaka head office staff then selected one story from the 16 submitted. The selected stories and reasons for selection were communicated back to the level below and the original storytellers. Over time the stakeholders began to understand the impact they were having and the project’s beneficiaries began to understand what the stakeholders believed was important. People were learning from each other. The approach, called Most Significant Change, systematically developed an intuitive understanding of the project’s impact that could be communicated in conjunction with the hard facts.

Rick’s method was highly successful: participation in the project increased; the assumptions and world views surfaced, helping in one case resolve an intra-family conflict over contraceptive use; the stories were extensively used in publications, educational material and videos; and, the positive changes where identified and reinforced."

Example taken from Anecdote.com: Evaluating the soft stuff

Advice

Advice for CHOOSING this approach(tips and traps)

  • MSC is particularly useful when you need different stakeholders to understand the different values that other stakeholders have in terms of "what success looks like" - criteria and standards for outcomes, processes and the distribution of costs and benefits.
  • MSC works best in combination with other options for gathering, analysing and reporting data. It doesn't provide comprehensive information about the impacts produced by an intervention. 

Advice for USING this approach(tips and traps)

  • Ensure the stories are not highjacked for other purposes such as for promotional material. Data can only be used for the original stated purpose, which in this case is evaluation unless other uses have been negotiated and agreed to at the time.
  • MSC is not a quick option.  It takes time and an appropriate project infrastructure to generate understanding and value clarification (identifying what people think is important). The full MSC process involves analysis of stories and sharing with both contributors and stakeholders, which  requires a programme with several structures in it (for example, local, regional and national project structures) and it needs to be repeated through several cycles.
  • There is scope to be innovative in this option. Your project may not have a hierarchical structure so there may be other ways of forming groups around which the stories can be discussed and the values identified.
  • It can be challenging to get engagement of the different groups involved in the process and to maintain their interest.  Don't have too many cycles of review. 
  • Other Skills Necessary: Good facilitation skills are important along with the ability to identify priorities.

Resources

Guides

The ‘Most Significant Change’ (MSC) Technique - A Guide to Its Use - Guidelines from the Rick Davies and Jessica Dart who developed the concept of Most Significant Change 

 

Strategy Development: Most Significant Change (MSC) - This guide to the Most Significant Change approach was written by the Overseas Development Institute (ODI). It provides an overview, a detailed description of the process, and an example of the technique in action.

 

Participatory Video and the Most Significant Change. A guide for facilitators - The toolkit is designed to support evaluators in planning and carrying out evaluation using Participatory Video with the Most Significant Change approach. 

 

Example

This overview from Capacity.org outlines the process and findings of an NGO self-evaluation process which used Most Significant Change to evaluate the organisational development services offered by the group.

 

 
A special thanks to this page's contributors
Contributor
Contributor
Research Fellow, RMIT University.
Melbourne.
Contributor
Director , Social Impact Consulting.
Australia.
Reviewer
Director of BetterEvaluation/ Professor of Public Sector Evaluation, Australia and New Zealand School of Government.
Melbourne.

Comments

Patricia Rogers's picture
Patricia Rogers

Thanks for these helpful comments, Theo.  We've updated the description and plan to add more detail in future about different variations of using MSC - some of which have cycles through levels of committees, and some which use other processes.  

Anonymous's picture
Bimal Dahal

Very helpful. Thanks. 

Anonymous's picture
Lara Bove

I can see how this approach can create a better understanding among stakeholders for the different views and values they hold. I'm thinking particularly of situations where one group of stakeholders holds most of the power and influence, and is unaware of just how limited their understanding is of the values or views of other stakeholders.

In fact, I think that sometimes that misunderstanding of other stakeholders is related to the stories one group imagines that the other group would tell. It makes so much sense to invite them to speak for themselves, and then listen so we can learn what they value and how they view things.

I also appreciate your caution about limiting the use of the stories for their intended purposes. It can definitely be tempting to use a positive account as a marketing tool, but we do have to honor people's trust and remember the context in which we asked a question.

Theo Nabben's picture
Theo Nabben

Dear Lara ,

I agree with your comments re MSC supporting the effective listening to people's voices about whats important. Re being careful about intended use of stories this is typically covered by proper ethics and needn't be complicated - in many development contexts we simply get permission (linked to collection guide) for potential intended uses and generally have some degree of confidentiality. 

Anonymous's picture
Mr R W Ebley

In order that change is effectively undertaken all levels of government and other public funded organisations need to demonstrate good management

I suggest independent management accreditation to achieve this

Thank you

Anonymous's picture
uneedmorequant

The authors are clearly not statisticians! On what basis does one determine the ‘most significant’ stories. How is ‘most’ defined? How is ‘significant’ defined?

As a purely qualitative method, I am surprised to see any reference whatsoever to terms such as ‘impact’.

This approach is described as not being able provide information about the usual experience but about the extremes. Why the focus on extremes?

What constitutes a ‘story’? Surely most qualitative techniques aim to gather information to form a narrative.

Patricia Rogers's picture
Patricia Rogers

You’ve raised some important questions about MSC.  Like any option or approach, it is useful for particular purposes.  MSC is particularly useful for identifying the different values of different stakeholders. The page has details of the processes used to identify what are seen as the most significant changes - meaning important, or valuable, not statistically significant.  Purposeful sampling of extreme cases is a good way to do this,

it is possible for qualitative data and non-counterfactual designs to be used for causal inference, although MSC does not do this apart from recording how key informants attribute cause.  If you check out the tasks in the section on the Understand Causes task in the Rainbow Framework, or search the site, you can find information on methods such as process tracing and qualitative comparative analysis ( QCA). 

Anonymous's picture
theonabben

Dear U need more quant

 Thanks for raising these points. You are correct - MSC does not attempt to follow a quantitative approach - it is qualitative and operates under guidelines to relative to qualitative not quantitative approaches. 

As Patricia signals significance and values is  something thats seem a most important at the time by the story teller or the selection panel. We manage this subjectivity by asking people to be transparent about why that particular story was most significant for them.These this will vary with context and overtime etc  See the comments from Rick Davies to a previous critique of subjectivity

Subjectivity matters! Peoples views of what is happening to them and other people matters, no matter what some objective observers might believe is actually happening. Their personal interpretations of what is happening affects their choices of what to do or not to do. A project would ignore their views at its peril.

Impacts aren't always easily measurable (depending on your definition and subject matter ) - you may appreciate Einstein's useful quote - "not everything the counts can be counted, not everything the can be counted counts" . You may find the DFID commissioned study by Stern et al on alternatives to traditional impact evaluation interesting. https://www.oecd.org/derec/50399683.pdf

One advantage of MSC is that it allows you to pick up unintended outcomes/impacts - something more difficult in a quant focus that typically looks to anticipated outcomes.

We consider extremes useful for highlighting insights and learnings.  other qualitative and quantitative approaches can inform us of the "normal or average" . We recognise MSCs limitations and hence recommend it use with other complementary evaluation techniques.  There many times MSC and quantitatve studies have been used within an evaluation because the complement each other. As Patricia signals - use the appropriate technique(s) to answer the question you want- this is the golden rule for choosing methods.  

MSC stories have a particular structure a beginning (before the change), middle (what happened) , end (the change that occurred) and explantion why story teller chose it as most significant. 

Hope this helps - Regards Theo

 

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.