Some nuts and bolts questions about coding
This guest blog by Helen Marshall springs from discussions of the Qualitative Interest Group (QIG) that Helen coordinates. QIG meets monthly in Melbourne Australia to discuss issues around researching with qualitative data. QIG members include those who are beginning to use qualitative data and experienced researchers.
This is a reflection on analysing qualitative data. While every project is unique, there are some things that all of us who use qualitative data to answer our research and evaluation questions have in common - we all need to ‘organise interpret and make meaning from the data’ as Michael Quinn Patton (2015) says. And we need to be able to defend our interpreting and meaning-making. In QIG, we have often talked about the challenges of analysing qualitative data and below I discuss some of the common questions that come up in our meetings.
When should you start thinking about coding?
The research question will impact on the analysis, shaping the things you look for and the way you seek connections, so you should think about coding as you think about the research question. No matter what method you are using, Start by building ideas for analysis into your design. Writing a personal reflection (some QIG members call them ‘memos’, others ‘notes’) on why you are doing the research, what you hope to see in the data and what you expect to see may be a good way to identify your own potential biases, and help you ensure that you set up ways to check your analysis as you go. Then as soon as you have some material (it might be your first round of observation or interviews, or it might be some literature you have read, or it might even be your personal reflection), start coding as a way of getting towards topics and concepts. (One of the regrets that we frequently hear in a QIG meeting is ‘I wish I had started coding as soon as I had any data’.)
How do you go from describing your data to interpreting it?
You usually want your coding to enable more than a bland description of your data. Starting with topics is a good way to ease into coding data and you could code all your data into a range of topic headings and then report on what you have. This will give you a descriptive evaluation, consisting of boring statements like ‘sixteen of the interviewees said that a good aspect of the service was that they felt listened to’.
To move beyond description to analysis you need to stop and think as you code. Lyn Richards (2015) suggests three steps as you code. First, see something and notice that it is worth coding (‘that’s interesting’). Next, stop and ask why it is interesting and maybe make a note about it (with ‘being listened to’, you might think it’s interesting because every service is supposed to hear their clients’ problems. Why are the clients here stressing the point?). The third step, the one that takes you towards analysis, requires you to think about why the snippet of data is interesting to you for this project, this often gives you the clue to how to name the category. Maybe ‘listened to’ is not enough, because the next phrase in the first snippet after ‘people listened to me’ is ‘they let me tell my story in my own way’. At this, you may start to think about ‘giving voice’ as a better way of expressing this, and that opens new possibilities.
What if you get stuck when coding and no categories seem to ‘emerge’?
If as you look at your material, you cannot see any ’themes’ or ‘analytic concepts,’ don’t worry. Start coding for ‘topics’ instead. It’s usually easy to see where someone is talking about the good aspect of service being that the providers listen to clients. You could code using those words. Later, you might see that listening to clients is an example of provider behaviour and create a hierarchy of provider behaviours including listening to clients. If you don’t want to code in detail at the start, you might just lump listening to clients under a broad heading of provider behaviour and later start teasing out its varied kinds.
Going beyond descriptive analysis sometimes demands imaginative leaps. At one QIG meeting, there was a strong recommendation of the shower as a good place to catch at the big picture that may elude us even after lots of reading of literature and lots of careful coding of data. In a small project I carried out a long time ago, expert researchers recommended other techniques for encouraging subconscious processing of material, including taking the dog for a walk and sitting under a tree and having a sleep (see Marshall 2002). Conversation about data and ideas is a good tool too - QIG members often say that the most valuable thing in our meetings is the chance to exchange views about their data with people not connected with their project!
What are some ways QIG members suggest managing data during coding and analysis?
There are lots of possibilities. You could use software, draw tables, work on index cards, use coloured pencils or post-it notes. Dedicated packages can be powerful and can make it easy to connect your coding with your ideas, but they take time to learn and are expensive to purchase. Excel tables where you can copy and paste snippets under column headings are simple to create, and you could connect your ideas and interpretations to your coding by adding columns but searching for patterns in a big set of data can be daunting. Word documents where you can type or copy and paste snippets of data into separate documents are another simple possibility, and you might be able to use tools like footnotes for adding your ideas but seeing patterns may be very difficult. Paper-based techniques like colour highlighting printed pages, using post-it notes or coding onto index cards can be very portable for small projects, but unwieldy with large amounts of material. Think in terms of your needs and the purposes of your research to set up a filing system that will work for you, remembering that you will probably want a system that allows for some alterations to be made as you go (Bazeley 2013 is an excellent resource full of ideas on how to work).
How do you build rigour into qualitative analysis?
QIG meetings often discuss how to get to trustworthy claims about our data and what it means. Practices like triangulation (of data, of methods, possibly of theories) can be built into research design. Insights from one body of data (client interviews) might need to be checked with other data (perhaps a focus group of the service providers). Data can be examined from more than one perspective and or theoretical lens.
Data and interpretation can be critically examined using the ‘null hypothesis’ approach. If you see what you think is a strong theme, pause for a moment. Posit the opposite: e.g. ‘giving voice is NOT important to clients’. Now think about what evidence there may be for that statement. Check it out in simple ways to start with, for example, just by counting the number of participants who are coded at ‘giving voice’ category. If it is a clear majority, your null hypothesis looks a bit rocky, and your initial view has some support. But wait! Did all the participants put a strong emphasis on giving voice, or was it just something they noted in passing… If your null hypothesis fails on this too, you have strong grounds for your initial thought. You can describe the grounds in the report so that your readers can accept your statement with confidence.
Memos and journals are another option to help with rigour, as they assist in making decision-making throughout the qualitative process transparent and traceable (see the new BetterEvaluation method page on memos and journals). There are many ways to get trustworthy claims about data and what it means. Taking a critical approach to your own interpretations by playing the ‘null hypothesis’ game is one. You should report how you have ensured rigour in your work.
The notes above show why, as Lyn Richard says and as QIG members constantly moan, ‘qualitative research is not a task to be hurried’. (2015 p.112). If you want a good evaluation using qualitative data, your very first step must be to design an appropriate project and leave yourself enough time to collect appropriate data while coding, re-coding, and refining your steps and ideas. You need to do rigorous coding, reflect on the coding and the analysis as it grows, always recording your processes and thoughts as you go (as well as leave time for your imagination to work on your data), test coding and analysis rigorously, and finally produce whatever kind of report is needed.
So, the short answers to the questions we talk about in QIG are as follows:
- Think about how best to manage your data - this will depend on your needs, the tools you can access and the purposes of your research
- Build in thinking about coding as you set up your project
- Start coding as soon as there is anything to code
- It’s fine to start coding by looking at topics but try to go beyond mere description
- Ask questions of your data as you code - is it interesting, why is it interesting and why is it interesting for this project
- Give your imagination a chance to help you move beyond description
- Make sure you build rigour into your analysis and report how you have ensured rigour in your work
What do you think?
One of the values of the QIG discussions (and other groups around the world that bring researchers or evaluators together to talk about methods) is hearing how people approach tasks differently. We'd love to hear how you approach qualitative data. Do you have any tricks for what to do when you get stuck and no categories seem to be emerging? What about advice for ensuring rigour?
Bazeley, P. (2013). Qualitative Data Analysis Practical Strategies Sage, London.
Marshall, H. (2002). 'What do we do when we code data?' Qualitative Research Journal 2:1 2002 pp56-70
Patton, M. Q. (2015). Qualitative research and evaluation methods 4th ed. Sage Thousand Oaks CA
Richards, L. (2015). Handling Qualitative Data, 3rd Ed Sage, London.