Week 34: Alternatives to transcribing interviews

Patricia Rogers's picture 29th August 2014 by Patricia Rogers

Being able to compare alternatives is essential when designing an evaluation. This post looks at some alternatives to transcribing interviews.

Last week a colleague asked me for some advice about her planned evaluation. She has just completed a pilot project which has been very successful. Now she wants to run the project again. The pilot project had extensive evaluation activity, including interviews with all participants. These interviews were very useful both on terms of the information it provided and the value of the reflective process itself for the participants. For the ongoing program, she wants to continue to conduct the interviews, because of the value of the process, but needs an alternative to recording and transcribing the interviews because the cost would be too much. What alternatives are there?

This is a common challenge in evaluation. While interviews are often used, often it is not possible to record and transcribe them for analysis: sometimes because of the cost of transcription or restrictions on recording the interviews sometimes waiting for a transcription will not provide timely information.  

A common strategy is for the interviewer to write notes during the interview, and then use these to write up the interviews. But there are considerable risks to validity in this strategy, especially for rich, semi-structured interviews which are not simply going through a list of questions and writing down short answers. There is no scope for transparency and review of what has been recorded, so its accuracy depends on how well the interviewer can simultaneously engage in an interview and take notes, the accuracy of their understanding of what is being said (harder in cross cultural interviewing) and their ability to overcome cognitive biases, such as finding it easier to remember things that fit with one's own views.

So are what are some reasonable alternatives to transcription?  Here are three I suggested to my colleague:

Sharing a summary of the interview with the interviewee at the end of the interview

At the end of the interview, take a few moments to go back through your notes with the person or people who have been interviewed, and say something like, "Do I have this right?  These seem to be the main issues you've raised…" and list them. This can be especially useful for interviews which have covered a lot of issues. For example, in a recent project, I was interviewing program managers about their experience of commissioning and managing an external evaluation. While I had specific questions to ask, I was mostly trying to understand their perspective on the experience, so they were encouraged to talk freely about their experiences and raise issues that were not on the interview schedule. In one interviews, when technical difficulties made it impossible to record the interview as planned, I made sure at the end of the interview to summarise what I understood to be the main points they were making, and invited them to correct or elaborate on it. 

Doing a graphic recording of the discussion

This can be particularly helpful in a group interview but can also be used in individual interviews. A graphic recording develops a visual summary of the conversation, such as showing different components, or alternatives, or dichotomies, or sequences. This can help others to share an emerging picture of what is being discussed, and then participants can be asked to validate, challenge or extend it. For a group interview, this can be done on a flip chart or a large sheet of paper along a wall. I sometimes do this on a sheet of paper when doing an individual interview.

Using a smart pen to provide audio of critically important sections

A smart pen records audio synchronised to notes taken during the interview.  This can significantly reduce the time needed to write up a good account of an interview. Most of the interview can be written up from the notes. Where there is a need to check details or get exact quotes, simply touching the pen to that section of the notes plays the audio.  This makes it easy to transcribe only key sections of the interview. If there is a need to check the whole interview, the audio file can be downloaded, shared and transcribed. A transcription service can provide a transcript of the file if needed. I have used a LiveScribe smart pen for interviews and it's been very useful, but it only works on special paper (which can be bought or printed out), and the smart pen needs special cartridges which are not always readily available, and of course it needs to be charged up and connected to computers. While it worked well while I was using it, the version I was using was not so easy to pick up and use again after a break.

What other alternatives to transcribing interviews have you used? Do you have other suggestions, or experiences of using the alternatives described above?

Image: Scribe, by jefka on Flickr

 

A special thanks to this page's contributors
Author
Director of BetterEvaluation/ Professor of Public Sector Evaluation, Australia and New Zealand School of Government.
Melbourne.

Comments

Patricia Rogers's picture
Patricia Rogers

Thanks for these additional suggestions - and your descriptions of when they are appropriate to use.  Deciding how to do evaluation does need to take account of the purpose - and the resource context.

b.harris-roxas@unsw.edu.au's picture
Ben Harris-Roxas

Some terrific ideas here, many thanks. I've found that a good alternative to LightScribe has been to use Evernote. It's not exactly the same but it allows you to record audio; take notes on a computer, phone or tablet; and even easily scan in handwritten notes or images. It's gotgood quite good integration with NVivo too. Another option to consider.

 

b.harris-roxas@unsw.edu.au's picture
Ben Harris-Roxas

Er, Livescribe. I think LightScribe is some CD labelling thing.

bob williams's picture
Bob Williams

Yeah but Evernote doesn't have the killer feature - the ability to identify a place in your notes and go straight to the audio.  Or at least I have not worked out yet how to do this.  There are some smartphone and tablet apps that do allow you to do this; Notability, SoundNote and Audionote.  All of them are lesso. All of them are less than $US5.   I haven't tried any of these alternatives to Livescribe, partly because the livescribe is so flexible and unintrusive to use.

bob williams's picture
Bob Williams

Oh something weird happened and only part of my note was posted.  Here is the full version :

Yeah but Evernote doesn't have the killer feature - the ability to identify a place in your notes and go straight to the audio.  Or at least I have not worked out yet how to do this.  There are some smartphone and tablet apps that do allow you to do this; Notability, SoundNote and Audionote are the main ones.  Notability and AudioNote has Mac versions also and Audionote has a PC version.   All of them are less than $US5.   I haven't tried any of these alternatives to Livescribe, partly because the livescribe is so flexible and unintrusive to use.

bob williams's picture
Bob Williams

The Australian action researcher and generally all round wise man, Bob Dick, has a fascinating process called Convergent Interviewing.  This is an rather novel form of interview design (or more accurately interview series or data collection design) that is beyond the scope of this discussion but implies a very restricted form of note taking.  

One feature is that at the end of the interview you ask the interviewee to summarise the key things you would want the interviewer to remember from the interview, while you stay quiet.  What happens is often very revealing.  I've learned to ask that question at about the two thirds stage of the interview rather than the 'end' because invariably the interviewee talks (often at great length) about what they wanted to talk about not what you wanted them to talk about.  

One of the other features of convergent interviewing is that you are only interested in aspects that either strongly confirm or strongly disconfirm any emerging themes.  All the other stuff in the middle is analytically not especially useful because it is messy, nuanced and we are deluding ourselves if we believe we can make sense of it.  Therefore why take notes on stuff you won't use?Essentially Bob's notion is that in practise, much as we may deny it, when we look back the basis of many of our conclusions and reporting come from the extremes, and that's all we need to record - and convergent interviewing has micro and meta processes that determine what that relatively limited data set is and how to record it.  Those who know Rob Brinkerhoff's Success Case Method and the various modifications suggested by Michael Scriven will recognise the same notions although their own approach implies considerable documentation.

If you want to know more about convergent interviewing, go to Bob Dick's web site :

http://www.aral.com.au/resources/iview.html

Patricia Rogers's picture
Patricia Rogers

Another suggestion, from @developmentWorx, is the C-Pen.  

"The C-Pen will highlight any text (hand written and typed) and instantly insert the text in applications such as Word as if you would have typed it yourself using the keyboard. This means that one can specifically highlight a text from a physical document, book, magazine, just about anywhere and the text will be transferred into the electronic document. This is great for extracting quotes without having to type the quotes directly." 

You can read their review of it here http://developmentworks.co.za/product-review-c-pen/ . Has anyone else used this?

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.