BetterEvaluation FAQ: How do you go about analyzing data that has been collected from respondents via a questionnaire?

Alice Macfarlan's picture 1st September 2016 by Alice Macfarlan

In this edition of the BE FAQ blog, we address a question that comes up quite often: How do you go about analysing data that has been collected from respondents via a questionnaire?

This is a good question – and a common one. Often people get to the end of having collected data from a survey and become somewhat stumped about what to do with it, often feeling overwhelmed by what can seem a daunting process if you are not used to it. So we’ve broken the key steps involved to hopefully make this process clearer – and easier.

An important part of analysing data from questionnaires is to go back to your broad Key Evaluation Questions (KEQs).  What do you want to know?  Do your analysis and report it in terms of these broad questions.  For example, you might want to know what participants’ experiences of a program have been like.  If you have questionnaire items where they have provided ratings (for example, 3 out of 5 for Relevance), show the frequencies of the data (how many rated it as 1, 2, 3, 4 and 5) either in a table or in a graph.  Check for the overall patterns – for example, is there a large group with high ratings and a large group with low ratings?  If so, do some crosstabulations to see if you can understand the pattern more – for example, did women generally rate the program high but men rated it low?  Or did older people find it more useful than younger people? 

For any qualitative data, see if you can group responses into themes and then report these themes with some illustrative examples. (See Thematic Coding)

Our Task Page - Analysis Data also has information about a number of techniques and resources for both textual and numeric analysis.  It also has a number of linked resource pages that discuss various tools for analysis data (e.g.NVivo or Excel resources). There's also a blog on using common software for analyzing data and some advice on combining qualitative and quantitative data here.

We have a number of resources specific to questionnaires linked to our Questionnaire Option Page - have a look through these as a first step, paying particular attention to any resources that give examples of how questionnaires have been analysed in previous evaluations, or guides which go into the analysis of questionnaires.

A special thanks to this page's contributors
Author
Australia.
Author
BetterEvaluation Website and Engagement Coordinator, BetterEvaluation and ANZSOG.
Melbourne, Australia.

Comments

Cris Sette's picture
Cristina Sette

Thanks for the blog Alice and Patricia.

I have used NVivo for some time now and it helped me greatly when looking at open-ended questions, and also analysis of large quantity of texts. But a common misunderstanding is that people think the program does the analysis for you. It does not, but help you organize the information, code it in a way that you become more consistent when reading text after text. As you read them, you, the analyst, code the text according to the evaluation questions or anything unexpected that shows up. At the end, the program helps you pull out all the parts about a particular theme and makes it easier to write your analysis, or quantify qualitative data. It is indeed a great resource. 

The last bit of my comment is that however very useful, the NVivo program is not that simple and some training is needed. But is worth and helps you with questionnaire analysis. 

Best,

Cris Sette, Process Facilitator

rickjdavies's picture
rick davies

This posting addresses an issue that has been of concern to me for some time. That is, people seem to spend more time planning how they will do data collection, relative to how they will do data analysis. Evaluation proposals should spell out how they will analyse data, not just how they will collect it.

Three types are worth considering: (a) univariate data, which is relatively straightforward, (b) bivariate data and (c) multivariate data. For every additional survey question asked the number of simple cross tabulations increases arithmetically. So fairly soon choices need to be made about what analysis to do and what not to do, simply because there will not be enough time to do everything. Those choices need to be transparent in their rationale. Ideally they would be made and spelled out in advance of data collection. Otherwise an evaluation is vulnerable to non-transparent ah hoc and opportunistic cherry picking of findings. The situation is even more demanding when considering any multivariate data analysis, where the number of possible combinations of question responses increases exponentially with each new question being examined.

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.