Hierarchical Card Sorting

Synonyms: 
HCS

Hierarchical Card Sorting (HCS) is a participatory card sorting option designed to provide insight into how people categorise and rank different phenomena. It focuses on a process of iterative comparisons between different subjects around a shared theme (decisions, plans, actors, etc), typically written on cards which are in turn sorted in line with the participant’s own value judgments, criteria and standards. Primarily intended for use with single respondents, HCS can be easily adapted to a small group setting, and is used as a central component in the creation of treemaps. The option was developed by Rick Davies and was influenced by the work of Gregory Bateson. 

The steps to hierarchical card sorting are:

  1. Identify the respondent’s area of expertise or knowledge that you want to explore. For example, a project manager’s knowledge of the locations in which a project is working, or the different activities being undertaken by the project. Or a donor’s knowledge of their grantees and their performance.
  2. Identify a list of relevant actual cases, e.g. of the project locations or activities. Try to avoid using too many (say 25), otherwise the task will become time consuming and too difficult.  Write down the name of each case on a separate card and number each card.
  3. Place all the cards in one pile and ask the respondent to tell you about some of the differences between all these cases. This is a warm up exercise that generates a shared awareness of the large number of differences that exist.
  4. Ask the respondent to sort all the cards into two piles of any size according to what they think is the most important difference between all the cases represented on the cards. If there is a particular issue that you want to explore then clarify the question “In your roles as…what do you think is the most significant difference between…?. Or “Considering the objectives of this organisation…what do you think is the most significant difference between…?
  5. Emphasise that a distinction is important if it makes a difference. Check the significance by asking “What difference does this difference make ?” If one can’t be identified then suggest to the respondent that they consider if there are other differences which might be more important.
  6. Record which cards are in which pile, the reported difference between the two piles, and what difference that difference makes.
  7. Take one of the two piles at time and repeat stages 4 to 6 above. Then repeat this process with the second pile. Repeat the same process until there is only one card left in each pile. If you have many cards to start off with you may decide to end when each group has 2 or less cards, or some other lower limit.

The results of the card sorting exercise is called a treemap, which can be summarised in Excel (rows = cases, columns = successive sorts, cell contents = differences), as shown in the example below. Consequences of the differences can be inserted as Comments in the relevant cell, or as different coloured text in the same cell as the difference.

For evaluation-based discussions, some additional steps are needed. After producing a treemap, call the participant’s attention to the results of the first sort and ask evaluative questions such as “which of these two groups were most successful?” Then move to the second level of sorting and ask the same question about the results of each of the two sorts at this level, then move to the next level and so on.  The answers will enable all the groups of cases to be sorted in rank order, from most to least successful. This process is extremely adaptable to whichever evaluative factors the investigator is interested in exploring. PS: In addition to asking about relative success, the answers can be followed up with a “Why do you think so” question. 

Example

In 1994 a senior CAA staff member in Bangladesh was interviewed about the NGOs that CAA was funding, using the HCS method. The treemap below shows the results. After the sorting exercise was completed he was asked which types of NGOs he would “most like to fund in the future”, starting with the first two groups on the left, and then progressing to the right. Green cells represent the preferred types in each set of two kinds of NGOs. The ranking on the right is evident from all the binary ranking choices. There is one set of tied ranks (5.5), where the respondent could not make a choice.

(Click image to enlarge)

The same treemap could be revised some time later, to seek judgements about the relative performance of the categories of NGOs

(http://mande.co.uk/special-issues/hierarchical-card-sorting-hcs/)

Advice

Advice for USING this option (tips and traps) 

  • It is important to stress to participants at the outset that their opinions are central to the exercise. As with all participatory options, trust and confidence between evaluator and participant are central to the process.
  • Keep the selection of cases which will be sorted to a manageable number (typically 25 or fewer) so as not to bore participants or create time-consuming work later.
  • Occasionally more than one case/example is left in a pile at the end of the exercise, which may be the result of a participant not being able to articulate any clear difference between them. Simply note this outcome and proceed to the next pile.
  • Be aware that people’s views are context- and time-dependent: the perspectives gathered in one HCS session may not remain constant in a later exercise. 
  • Participants may respond that they can see no difference between a particular set of cases. In these situations, the evaluator should emphasise that differences can be found in even seemingly-identical examples, and that their interest is in determining relative, subjective differences from the perspective of the participant, as opposed to absolute, objective divisions.
  • If the participant offers too many distinctions that do not seem significant, ask “In what way is that significant, what difference does that make?” This can check whether the respondent can articulate the significance, and whether they understand the exercise.
  • If the participant offers many distinctions between two cards, make sure to ask which is the most significant and why.
  • Remember to ask the differences these differences make. In this way, you will surface working, and often tacit, assumptions or hypotheses

Resources

Tools and Examples

Sources

Rick Davies (1996). Hierarchical Card Sorting: A Tool for Qualitative Research, Centre for Development Studies, Swansea UK. Swansea University. Retrieved from http://www.mande.co.uk/docs/hierarch.htm

Rick Davies (1998). Tree Maps: A Tool for Structuring, Exploring and Summarising Qualitative Information  Centre for Development Studies, Swansea UK. Retrieved from http://www.mande.co.uk/docs/treemap.htm

Harloff, J. and Coxon, A.P.M. (2009) How to Sort: A Short Guide on Sorting Investigations. Retrieved from http://www.methodofsorting.com/HowToSort1-1_english.pdf See also their

Also from http://mande.co.uk/

Feature image: Photograph by Ana María Guardia (2009). Actividad de card sorting. Retrieved via https://www.flickr.com/photos/palabrerio/3861425763. Licensed under Attribution 2.0 Generic (CC BY 2.0)

Updated: 22nd September 2016 - 10:53am
A special thanks to this page's contributors
Author
Monitoring and Evaluation Consultant, MandE NEWS.
United Kingdom.
Contributor
Oxford University.
Oxford.

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.