Excerpts from David F. Harris’s New Book

Portions of the following are excerpted from, The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions, by David F. Harris.

The Framework for Writing Questionnaires

My purpose in writing this book is to provide a user-friendly, comprehensive guide for planning and writing questionnaires. This book offers a framework for writing questionnaires that begins with planning research to support decision-making, conducting qualitative research, and planning the questionnaire before you begin writing questions. I then provide 65 guidelines for making questions clear, answerable, easy, and unbiased. I also provide guidelines for the three most common tasks we ask of respondents: selecting items from a list, rating items on a scale, and answering open-ended questions. Finally, I discuss how to properly pretest a questionnaire.

The Framework

Example Guidelines

Each section of The Framework has guidelines for how to write questionnaires. Here is an example from Make Questions Clear.

A Brief Example of Using Guidelines to Improve a Question

Questionnaire writing is at least as difficult as any other form of writing. The guidelines provided will help you communicate better with each respondent. A better questionnaire provides more accurate data and, ultimately, supports better decision-making for your organization.

Finally, virtually all the questions you will attempt to write for your questionnaires will benefit from the application of several guidelines. Consider the following first draft of a question:

Q: How frequently do you go to the grocery store?

___ Very frequently

___ Frequently

___ Neither frequently nor infrequently

___ Infrequently

___ Very infrequently

This seemingly simple question contains several significant pitfalls. First of all, it lacks a unit of measurement; frequency is a vague term that has a different meaning to different people (Clear #1). The question also lacks the time frame in which you want people to recall the information you need (Answerable #1). How about this revised question:

Q: In the past 30 days, how many times have you gone to the grocery store? _______

This is better. We have replaced “frequency” with “how many times” (a countable concept) and added a time frame of 30 days. Now, we need to clarify the referent of “you” (does it refer to the generic you or to the respondent?) (Clear #4) and label the answer box to make clear that we want a single number, not a range of numbers (e.g., 5–10) or a text response like “ten” or “a lot.”

Q: In the past 30 days, how many times have you, yourself, gone to the grocery store?

______ # times gone to grocery store

We also need to think carefully about the concept grocery store (Clear #3). Would we want respondents to include visits to gas stations or drugstores to pick up a few grocery items? What information do we really need (Questionnaire Planning #2)?

Q: In the past 30 days, how many times have you, yourself, gone to the grocery store to purchase groceries? (Please do not include visits to gas stations or drugstores to pick up food items.)

______ # times gone to grocery store

Most of the examples in this book illustrate one or two problems that could be fixed using only one or two guidelines. For learning purposes, that made the most sense.

When you write questionnaires from scratch, however, you will need to keep all of these guidelines in mind. Using all the guidelines associated with planning research, planning the questionnaire, and writing questions will help create a good questionnaire. In part four, we will explore the final step—using the guidelines associated with pretesting—in making the questionnaire truly excellent.

Proper Pretesting

Properly pretesting a questionnaire is critical. Here is an example of one approach. Below are instructions that you would give each respondent at the beginning of the pretest, followed by the first two questions with proactive probes and one possible reactive probe.

Instructions: Thank you for coming in today. Let me tell you about what we are going to do. We are testing a draft questionnaire that will eventually be given to people like you, after we improve it. We need to find out if the questions make sense and if people can answer them. I am going to ask you to read each question and answer it, just like a regular questionnaire. After you read each question and answer it, I am going to ask you a few questions.

Q1. In the past 12 months, how many times have you suffered a migraine headache?

____ # migraine headaches in past 12 months

Proactive Probes:

1. In your own words, what do you think this question is asking?
2. Tell me how you came up with your answer.
3. Was it easy or hard to come up with this answer?

What does the phrase, “suffered a migraine headache,” mean to you? (PROBE on suffered or migraine if necessary)

Q2. In the past 12 months, how many times have you seen a doctor about your migraine headaches?

____ # times seen doctor about migraine headaches in past 12 months

Proactive Probes:

1. Please rephrase this question in your own words.
2. Tell me how you came up with your answer.
3. How easy or hard was it?

Possible Reactive Probes:

1. I noticed you laughed when you said “doctor.” Why?
2. You mentioned that you called your doctor about your most recent migraine but did not see him in person. What if the question asked “In the past 12 months, how many times have you talked to your doctor about your migraine headaches, either in person or by phone?” Would that work?

Summary

I wrote this book to help you write better questionnaires. But my ultimate purpose is the same as the underlying purpose of any questionnaire: to help you and your organization get better information for better decision-making.

David Harris conducts training and consulting on questionnaire design and research planning. He has twenty years of experience designing and implementing qualitative and quantitative research in numerous categories. He received his B.A. from Reed College, in Portland, Oregon, and his M.A. in Quantitative Psychology from the L. L. Thurstone Psychometric Laboratory at the University of North Carolina at Chapel Hill.