Questionnaires

Synonyms:
Surveys, survey

A questionnaire is a specific set of written questions which aims to extract specific information from the chosen respondents. 

The questions and answers are designed in order to gather information about attitudes, preferences and factual information of respondents.

Questionnaires focus on the sampling of a smaller group of people that are statistically representative of the wider population in question. This sample, in turn, proves more manageable to study, reducing the investigator’s overall workload and costs while also making it easier to ensure homogeneity and quality within a smaller data-set. Three primary categories of surveying sampling include probability/random sampling (in which every unit of the population has an equal chance of being included in the sample) and non-probability/non-random sampling (in which certain elements of the population are intentionally excluded from the possibility of being part of the sample, for example by 'convenience sampling'). Stratified sampling also involves the identification of sub-populations from the overall population which are then sampled randomly. 

Once investigators have drawn their sample population, questions are administered to respondents. A number of data collection approaches can be followed to collect answers from the sample. Each approach has competing advantages and disadvantages and may be more or less appropriate depending on the context, cost, coverage offered of the sample population, flexibility, accuracy and anticipated response rates. Note that approaches can be combined. Some of the most common include:

Questions asked in surveys are systematic – the same inquiries are asked of each and every respondent, in the same format and order - and are typically presented in the form of a questionnaire or structured interview. Questions can be either open or closed-ended (although open-ended questions are often coded into quantitative response scales during analysis). More specifically, questions can take one of several forms:

  • Dichotomous: respondents chose between two options for response
  • List: respondents chose between a list of more than two items, of which any can be selected
  • Category: respondent chooses between one of several mutually exclusive categories (such as age ranges, i.e. 20-29, 30-39, etc).
  • Ranking: respondents are asked to place several items into a ranked order
  • Quantity: the respondent is asked to provide an exact or approximate number
  • Grid: respondents are presented with a table or grid in order to answer two or more questions simultaneously
  • Verbal: respondents are asked to provide verbal word, phrase, or comment responses
  • Rating scales: respondents are asked to rate items or phrases against pre-set criteria. The most common rating scale is the Likert scale which asks for the level of agreement on a symmetric agree-disagree scale for a statement.

Conducting a survey itself is a serious undertaking, one which demands careful preparation across three general stages: preparation and question design; administration of the survey, and analysis of results.  Staff will need to be carefully trained and prepared for the task, and all relevant administrative and logistical preparations should be well sorted before the survey’s start. Questionnaire design must be given particular attention – the scope, content and purpose of individual questions need to be determined, a proper response format selected, and care given to the question’s specific wording, formatting, and ordering. If at all possible, questionnaires should always be pilot-tested before being sent out en-mass.

One key aspect of surveying is how to maximise response rates among the population. There is a universe of specialised tips and techniques per approach, but some broad general tips to increase the likelihood of those contacted actually responding include:

  • Sending advance letters/calls/e-mails to respondents which presents the investigator’s research agenda and details of the project, when the respondent can expect to be contacted by call or by an in-person visit, and thanking the respondent for his or her potential time.
  • Construct ‘respondent-friendly’ survey questionnaires – as brief as possible (ideally single page), clearly written, easy to complete, and unlikely to cause offence.
  • Survey staff should be carefully trained and briefed on both the research project itself, as well as specialised training for the administration of interviews and group questionnaires, scheduling, and logistics.
  • Offer incentives, if appropriate – financial or non-monitory incentives given at the start or completion of a survey can help response rates, but should be carefully considered for any ethical complications this might raise.
  • Appealing to people emotionally – such as emphasizing the potential to make a positive impact on a social issue by taking part in a survey – can often persuade reticent or unmotivated respondents to participate. Likewise, appeals made by authorities can have a similar effect, such as legal obligations to complete government census forms or required participation in a survey as an official policy of an organisation.

Source: Research Methods Knowledge Base (2006).

Examples

UNICEF’s Multiple Indicator Cluster Survey (MICS) is a household survey programme developed to assist countries in generating data on the livelihood situation of women and children (UNICEF, 2015). Several rounds of the MICS has been administered worldwide by UNICEF since 1995; the current round (MICS4) was recently finished and involved national-level surveys in 40 countries around the world. Three MICS4 questionnaires were developed as research instruments for this survey: a household questionnaire, a questionnaire for individual women, and a questionnaire for children under five.

Below is a sample page of the household questionnaire. (Each answer includes directions in the third column for proceeding to the next question.)

UNICEF handwashing questionnaire

Source: UNICEF. (2010) “Household Questionnaire”, Multiple Indicator Cluster Survey/MICS4.

Advice for choosing this method

  • Surveys are major undertakings: before you start, be sure you have the resources, time, staff, expertise and time-management skills necessary, or you can easily become quickly overwhelmed.
  • Surveys are particularly useful for collecting large amounts of data in a relatively short time and are less expensive than many other data collection techniques. They can also be administered over long distances by phone, computer, and mail, thus allowing the investigator to be based elsewhere if necessary. Finally, surveys are the most useful method for describing a large population’s characteristics with statistical significance.
  • On the other hand, surveys have several notable disadvantages. First, without careful survey construction and administrative oversight, even well-designed studies can quickly fall apart. Second, there will always be the potential for respondents to lie, suffer memory gaps, or feel obligated to answer in a certain way to impress the investigator – as such, their answers do not always reflect reality. Third, response rates will always retain a potential for bias, because people who chose to respond can be different from those who chose not to.
  • In selecting the particular survey method, consider. First, whether the population of interest can be enumerated (that is, can you locate the people you wish to question through preexisting data, like phonebooks, registries, etc, or will you need to search them out yourself?). Second, are there any language or technology-access issues which might make certain methods inappropriate (for example, administering an online survey in an area of low computer usage). Third, whether the population will be willing to cooperate with your research (for example, asking about anti-government sentiment in a repressive state). Fourth, whether your population of interest is so geographically dispersed as to make certain methods (such as door-to-door in-person surveys) unrealistic.

Each data collection method has advantages and disadvantages. For several of the most common:

  • Mail-based questionnaires are low cost, respondents are able to answer at their convenience, there is no interviewer bias, and they can be far longer than other methods. On the other hand, mail-based questionnaires are a poor format for asking any question that could require clarification from the respondent, and there are often significant delays before they are returned by post.
  • Telephone questionnaires often result in higher response rates (people are more willing to speak to another person directly and in the moment), but can also neglect large aspects of your population of interest in areas of low-phone usage. They are also incompatible with any questions that rely on visual prompts, such as the interpretation of pictures or graphs.
  • Internet questionnairesare low cost, offer immediate feedback, and given that respondents enter their answers directly, frees investigators from a large component of data entry. However, they are restricted to computer-literate respondents and can be easily manipulated by single respondents who complete the survey multiple times under aliases (a problem that can be addressed to some degree through password protection).

Advice for using this method

In designing a questionnaire, investigators should:

  • Be sure to obtain any official approval needed to distribute the questionnaire.
  • Carefully consider the timing of the survey stages, and set milestones to monitor progress.
  • Establish a system in advance to handle paperwork and record information on responses, non-responses, area coverage, etc. which will rapidly accumulate.
  • Plan for follow-up surveying to be done on non-respondents, in order to maximise coverage.
  • Consider the analysis stage early on, and how your survey design will impact the subsequent analysis. This will help ensure you only collect data that will be used.
  • Speak to any colleagues or peers who have experience in surveying your population of interest – there are often numerous, idiosyncratic details of survey design that can only be learned from practical experience working with a particular population.

In constructing questionnaires:

  • Link every question to a broader research agenda, and ensure that there are no superfluous questions.
  • Consider your question’s content: can the respondents be realistically expected to know the answer? Will they be able to answer in the moment, or will they need to consult records to provide an answer?
  • Be sure to leave the respondent the option of responding with “I don’t know” or “Not sure” to an answer.
  • Use jargon-free, technical-free language, unless your population of interest is made up of experts.
  • Specific questions are better than vague ones – for example, “Did you walk to town this weekend” rather than “Did you walk to town?”.
  • The order of the questions is extremely important – ask easy questions at the beginning, and avoid more sensitive or demanding questions for later in the questionnaire.
  • Requisite background information which should always be included at the start of a questionnaire includes your research affiliation and purpose, the return address and date, a clear statement of the confidentiality and anonymity you are offering, as well as the voluntary nature of the survey (if it is not a legal requirement), and a note of thanks for the respondent’s time. Most importantly, you must include careful instructions for filling out the questionnaire, including exact details and, ideally, examples.
  • All questionnaires should have a designated number including information on the date and location administered.
  • Ensure the questionnaire has an attractive layout and formatting, which will aid in both response rates and in your subsequent analysis.
  • Leave enough time to plan, design, and pilot-test first drafts of a questionnaire for revisions, as well as enough time to print the required materials (if paper-based).
  • Avoid any potentially offensive content. Be aware and sensitive of the local context – cultural, religious, educational level, etc.

Resources

Guides

Tools

Methods

Research Methods Knowledge Base (2006), Survey Research, Retrieved from: http://www.socialresearchmethods.net/kb/survey.php

UNICEF (2015). Multiple Indicator Cluster Survey (MICS). Retrieved from: http://mics.unicef.org/tools

Expand to view all resources related to 'Questionnaires'

'Questionnaires' is referenced in: