Questionnaire Design

“It is not every question that

deserves an answer.”
Publius Syrus
(roman,1st century B.C.)
A QUESTIONNAIRE
IS ONLY AS GOOD
AS THE QUESTIONS
IT ASKS
 To understand the role of the
questionnaire in the data
collection process.

THE
THEROLE
ROLEOF
OFAA Questionnaire Defined
A set of questions designed to generate the data necessary for accomplishing the objectives of the research project.
QUESTIONNAIRE
QUESTIONNAIRE The Critical Link
Positioned between the survey objectives and the respondent’s information.
Translates the objectives into specific questions to solicit information.
A Good Questionnaire Appears
• As easy to compose as a good poem
• But, it is usually the result of long, painstaking work
The Major Decisions in
Questionnaire Design
1. What should be asked?
2. How should each question be phrased?
3. In what sequence should the questions be
arranged?
4. What questionnaire layout will best serve the
research objectives?
5. How should the questionnaire be pretested?
Does the questionnaire need to be revised?
Phrasing Questions
• Open-ended questions
• Fixed-alternative questions
 Classifying Surveys
by Degree of Structure and Degree of Disguise
Structured Unstructured

Example: Example:

Typical descriptive survey Survey with open-ended

Undisguised with straight-forward, structured questions to discover “new”
questions answers or focus group interview

Disguised Example: Example:

Survey interview to measure Projection techniques used

brand A’s image versus mostly for exploratory research
competitive brands’ images or
brand recall (unaided recall)
Developing a Questionnaire
• No hard and fast rules
• Only guidelines
• Avoid Complexity: use simple, conversational
language
• Avoid leading and loaded questions

• Avoid ambiguity: be as specific as possible

• Avoid double-barreled items

• Avoid making assumptions

• Avoid burdensome questions

Figure The Questionnaire’s Role in the Research
Process

Survey Objectives Respondent

Information

Questionnaire

Data
Analysis

Findings

Recommendations

Managerial
Action
 To become familiar with the
criteria for a good questionnaire.

Does it Provide the Necessary Decision-Making Information?

CRITERIA
CRITERIAFOR
FORAAGOOD
GOOD If the questionnaire fails to provide insights, then discard or revise.

QUESTIONNAIRE
QUESTIONNAIRE Does It Fit Respondent Requirements?
What About Editing and Data-Processing Requirements?
A Questionnaire Serves Many Masters
Figure 2
The Questionnaire Development Process

(10)
(2)
Implementation Data Collection
(9) Methods
Final Copy

(3)
Question Response
(8) (1) Format
Pretest and Revise Survey Objectives

(4)
(7) Question Wording
Obtain Approval

(6)
Evaluate Layout
(5)
Flow and Layout
 To learn the process for
questionnaire design.

Step One: Determine Survey Objectives, Resources, and Constraints

THE
THEQUESTIONNAIRE
QUESTIONNAIRE The information sought.
Step Two: Determine the Data Collection Method
Impacts questionnaire design.

DEVELOPMENT
DEVELOPMENT Step Three: Determine the Question Response Format
• Open-ended Questions

PROCESS
PROCESS
• Close-ended Questions
 To learn the process for
questionnaire design.

• Dichotomous Questions
THE
THEQUESTIONNAIRE
QUESTIONNAIRE • Multiple Choice Questions
DEVELOPMENT
DEVELOPMENT • Disadvantages of Closed-Ended Questions
• Scaled-Response Questions
PROCESS
PROCESS
TYPES OF QUESTION

DICHOTOMOUS YES/NO

MULTIPLE ONE/MANY
CHOICE RESPONSES

ODD/EVEN
RATING SCALES
NUMBERS

OPEN-ENDED FREE RESPONSE

TYPES OF QUESTION

RANKING 1st/2nd. ETC.

MANY
RATIO DATA
RESPONSES

DISTRIBUTING
CONSTANT SUM
MARKS

MARKS OUT
RATIO DATA
OF TEN
 Step Four: Decide the Question Wording To learn the process for
1. The wording must be clear.
2. Select words so as to avoid biasing the respondent. questionnaire design.
3. Consider the ability of the respondent to answer the question.
4. Consider the willingness of the respondent to answer the question.

THE
THEQUESTIONNAIRE
QUESTIONNAIRE
DEVELOPMENT
DEVELOPMENT
PROCESS
PROCESS
 Step Five: Establish Questionnaire Flow and Layout
To learn the process for
1. Use the screener questions to identify qualified respondents.
questionnaire design.
2. After obtaining a qualified respondent, begin with a question that
obtains a respondent’s interest.

THE
THEQUESTIONNAIRE
QUESTIONNAIRE 3. Ask general questions first.
DEVELOPMENT
DEVELOPMENT 4. Ask questions that require “work” in the middle of the
PROCESS
PROCESS questionnaire.
 To learn the process for
questionnaire design.

THE
THEQUESTIONNAIRE
QUESTIONNAIRE 5. Insert “prompters” or strategic points.
6. Position sensitive, threatening, and demographic questions at the end.

DEVELOPMENT
DEVELOPMENT
Step Six: Evaluate the Questionnaire and Layout
• Are the Questions Necessary?
• Is the Questionnaire Too Long?
PROCESS
PROCESS • Will the Questions Provide the Desired Information to Accomplish the Research Objective?
 To learn the process for
questionnaire design.

THE
THEQUESTIONNAIRE
QUESTIONNAIRE • Allow Plenty of Space for Open-Ended Responses
• Printed Instructions Capital Letters

DEVELOPMENT
DEVELOPMENT
Step Seven: Obtain Approval of All Relevant Parties
Distribute copies to all parties with authority
Step Eight: Pretest and Revise
PROCESS
PROCESS Use the best interviewers
 To learn the process for
questionnaire design.

THE
THEQUESTIONNAIRE
QUESTIONNAIRE Step Nine: Prepare Final Questionnaire Copy
Precise typing, instructions, spacing, numbering, and precoding must be set up, monitored, and proofread.
Step Ten: Implementing the Survey

DEVELOPMENT
DEVELOPMENT Supervisor’s Instructions
Interviewer’s Instructions

PROCESS
PROCESS
Call Record Sheet
Field Management Companies
 How software and the Internet
impact questionnaire design.

THE
THEIMPACT
IMPACTof
of
Software for Questionnaire Development
Senses Multimedia Researchers incorporates:
• sounds

SOFTWARE
SOFTWARE
• images
• animations
• movie clips
• earlier responses

TECHNOLOGY
TECHNOLOGYAND
AND Survey Said
• PC and Web based surveys

THE
THEINTERNET
INTERNETon
on
QUESTIONNAIRE
QUESTIONNAIRE
DEVELOPMENT
DEVELOPMENT
 How software and the Internet
impact questionnaire design.

INTERNET
INTERNETSELF-
SELF- SurveyBuilder.com works as follows:
1. Researcher creates custom survey at web site.

SERVICE
SERVICE
2. Instructions are e-mailed to link survey to web site.
3. SurveyBuilders.com “invites” participants.
4. Respondents complete survey - SurveyBuilder compiles data.
QUESTIONNAIRE
QUESTIONNAIRE 5. Users can remotely access survey results.

BUILDERS
BUILDERS
 To understand the role of
questionnaire design in data
collection costs.

THE
THEQUESTIONNAIRE’S
ROLE
QUESTIONNAIRE’S
ROLEIN
INCOSTS
COSTSAND
AND
Impact on costs and profitability:
PROFITABILITY
PROFITABILITY
• the questionnaire
• incidence rate
• termination during an
interview
• subject matter
• redundant or difficult
• questionnaire length
• changing the subject
Questionnaire General
Format

• Self-administered (by mail or

personal contact)
• In person (face-to-face)
• Telephone interviews
The Questions or Items
• Are the focus on any survey or
questionnaire
• It is crucial to know how to ask the
questions in written and spoken
form
• The way you ask the questions
determines the answers
Questions Context
• Identify the questionnaire’s specific
purpose(s)
• Clarify the terms used to state the
questionnaire’s purpose(s)
• Be sure to have the specific
objectives of the questionnaire
• Know the respondents
 Questions Context
• Standardize the interviewer
• Standardize the response format
• Ask questions in a social, cultural, and
economic context
• Keep confidentiality and voluntary
participation
• Include a letter of introduction or
presentation
 Questions
Format
• Open questions - more
information but difficult to codify,
enter, and analyze
• Closed questions - less information
but easy to codify, enter, and
analyze
How To Obtain
Valid
Information
• Ask purposeful questions
• Ask concrete questions
• Use time periods based on
importance of the questions
• Use conventional language
How To Obtain
Valid Information
• Use complete sentences
• Avoid abbreviations
• Review questions with
experts and potential
respondents
• Use shorter questions
 How To Obtain
Valid Information
• Avoid two-edged questions
• Avoid negative questions
• Adopt/adapt questions used
successfully in other
questionnaires
Question Types
• Questions are designed as either
• Open-ended
• Try to anticipate the response you will get
• Well suited for getting opinions
• Useful in explanatory situations
• Closed
• Use when all the options may be listed
• When the options are mutually exclusive
APPROACHING THE PLANNING OF A
QUESTIONNAIRE
• Stage One: Decide the purposes/objectives/ research
questions;
• Stage Two: Decide the population and sample
• Stage Three: Itemize the topics/constructs/ concepts;
• Stage Four: Decide the kinds of measures or
responses needed;
• Stage Five: Write the questionnaire items;
• Stage Six: Check that each research question has
been covered;
• Stage Seven: Pilot the questionnaire and refine;
• Stage Eight: Administer the questionnaire.
 OPERATIONALIZING A
QUESTIONNAIRE

• Clarify the questionnaire’s general purposes and then translate them

into a specific, concrete aim or set of aims.
• Identify and itemize subsidiary topics that relate to its central
purpose.
• Formulate specific information requirements relating to each issue.
• Plan with the data analysis in mind.
Types of Questionnaire-STRUCTURED, SEMI-
STRUCTURED AND UNSTRUCTURED
QUESTIONNAIRES
• The larger the size of the sample, the more structured, closed and
numerical the questionnaire may have to be.
• The smaller the size of the sample, the less structured, more open
and word-based the questionnaire may be.
• Structured questions take a lot of time to set up but then a short
time to process and analyze.
• Open questions take a shorter time to set up but a longer time to
process and analyze.
THE LAYOUT OF THE QUESTIONNAIRE
• It must look easy, attractive and interesting.
• Keep it as uncomplicated as possible.
• Clarity of wording.
• Simplicity of design.
• Simple, short and clear instructions for completion.
• Avoid placing instructions at the bottom of a page.
• Break down long lists of numbered items into separate sections,
each item in the section starting with the number ‘1’.
• Make it clear if respondents are exempted from completing certain
questions or sections (filter), and where they go next if they are
exempted.
• Include a preliminary statement of anonymity/confidentiality.
• Place response categories to the immediate right of the text.
 PURPOSES OF PILOTING
• To check clarity of items/layout/sections/presentation/
instructions;
• To gain feedback on appearance;
• To eliminate ambiguities/uncertainty/poor wording;
• To check readability;
• To gain feedback on question type (suitability/feasibility/ format
(e.g. open/closed/multiple choice);
• To gain feedback on appropriateness of question stems;
• To generate categories for responses in multiple choices;
• To generate items for further exploration/discussion;
• To gain feedback on response categories;
• To gain feedback on length/timing (when to conduct the data
collection as well as how long each takes to complete (e.g. each
interview/questionnaire))/coverage/ease of completion;
 PURPOSES OF PILOTING
• To identify redundant items/questions (those with little
discriminability);
• To identify irrelevant questions;
• To identify non‑responses;
• To identify how motivating/non‑engaging/threatening/
intrusive/offensive items may be;
• To identify sensitive topics and problems in conducting
interviews;
• To test for inter‑rater reliability;
• To minimise counter‑transference;
• To gain feedback on leading questions;
• To identify items which are too easy/difficult/complex/ remote
from experience;
• To identify commonly misunderstood or non‑completed items.
PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
• Operationalize the purposes of the questionnaire.
• Be prepared to have a pre-pilot to generate items for a pilot
questionnaire, and then be ready to modify the pilot
questionnaire for the final version.
• If the pilot includes many items, and the intention is to
reduce the number of items through statistical analysis or
feedback, then be prepared to have a second round of
piloting, after the first pilot has been modified.
• Decide on the most appropriate type of question –
dichotomous, multiple choice, rank orderings, rating scales,
constant sum, ratio, closed, open.
• Ensure that every issue has been explored exhaustively and
comprehensively; decide on the content and explore it in
depth and breadth.
• Use several items to measure a specific attribute, concept or
issue.
PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
• Ask more closed than open questions for ease of analysis (particularly
in a large sample).
• Balance comprehensiveness and exhaustive coverage of issues with
the demotivating factor of having respondents complete several
pages of a questionnaire.
• Ask only one thing at a time in a question. Use single sentences per
item wherever possible.
• Keep response categories simple.
• Avoid jargon.
• Keep statements in the present tense wherever possible.
• Strive to be unambiguous and clear in the wording.
• Be simple, clear and brief wherever possible.
• Clarify the kinds of responses required in open questions.
PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
• Balance brevity with politeness, e.g. replace a blunt phrase like
‘marital status’ with a gentler ‘please indicate whether you are
married, living with a partner, or single....’.
• Ensure a balance of questions which ask for facts and opinion.
• Avoid leading questions.
• Try to avoid threatening questions.
• Do not assume that respondents know the answers, or have
information to answer the questions, or will always tell the truth
(wittingly or not)..
• Avoid making the questions too hard.
• Balance the number of negative questions with the number of
positive questions.
• Consider the readability levels of the questionnaire and the reading
and writing abilities of the respondents.
• Put sensitive questions later in the questionnaire.
• Intersperse sensitive questions with non-sensitive questions.
PRACTICAL CONSIDERATIONS IN
QUESTIONNAIRE DESIGN
• Be very clear on the layout of the questionnaire so that it is
unambiguous and attractive.
• Avoid, where possible, splitting an item over more than one page,.
• Ensure that the respondent knows how to enter a reply to each
question.
• Pilot the questionnaire.
• With the data analysis in mind.
• Decide how to avoid falsification of responses.
• Be satisfied if you receive a 50 per cent response to the
questionnaire
• Decide what you will do with missing data.
• Include a covering letter.
• If the questionnaire is going to be administered by someone other
than the researcher, ensure that instructions for administration are
provided and that they are clear.
METHODS OF ADMINISTERING
QUESTIONNAIRES
• Post
• Self-administration in presence of researcher
• Self-administration without researcher present
• In situ completion (e.g. workplace/ home)
• Face-to-face interview
• Telephone
• Internet
 PROCESSING QUESTIONNAIRE DATA

• Coding
• Data reduction techniques
• Editing and cleaning data
Strategic Concerns in
Instrument Design

What type of scale is needed?

What communication approach will be used?

Should the questions be structured?

Should the questioning be disguised?

Technology Affects
Questionnaire Development
Write questionnaires
more quickly
Create visually driven
instruments
Eliminate manual data
entry
Save time in data analysis

WebSurveyor used to write an instrument.

Disguising Study Objectives

Reluctantly shared, Willingly shared,

Conscious-level Conscious-level
information information
Situations
Situations
where
where
disguise
disguise is
is
unnecessary
unnecessary
Knowable,
Subconscious-level
Limited-conscious-
information
level information
Question Content

Should this question be asked?

Is the question of proper scope and coverage?

Can the participant adequately

answer this question as asked?

Will the participant willingly

answer this question as asked?
Question Wording

Shared
Adequate vocabulary Single
alternatives meaning

Criteria
Criteria
Misleading
Personalized assumptions

Biased
Response Strategy

Objectives
of the study

Participant’s Participant’s
motivation to level of
share information
Factors
Factors

Ease and clarity with Degree to which

which participant participants have
communicates thought through topic
TYPES OF QUESTION

• Open to closed
• Choose the metric (scale of data):
• Nominal
• Ordinal
• Interval
• Ratio
• Do not assume that respondents have the
information/knowledge/views
LIKERT SCALES
• Useful for measuring degrees of intensity of feeling;
• No assumption of equal intervals;
• No assumptions of matched intensity of feeling;
• No way of knowing if respondents are telling the truth;
• No way of knowing if there should be other categories or
items;
• Halo effect;
• Allows for different scaling and mid-points, e.g.:
(a) strongly disagree – neither agree nor disagree –
strong agree;
(b) not at all – a very great deal;
• Central tendency;
• Ordinal data.
Free-Response Strategy

What factors influenced your enrollment in Metro U?

____________________________________________
____________________________________________
 Dichotomous Response
Strategy

Did you attend the “A Day at College” program at Metro U?

Yes
No
Multiple Choice Response
Strategy

Which one of the following factors was most influential

in your decision to attend Metro U?

Good academic standing

Specific program of study desired
Enjoyable campus life
Many friends from home
High quality of faculty
Checklist Response Strategy

Which of the following factors influenced

your decision to enroll in Metro U? (Check all that apply.)

 Tuition cost
 Specific program of study desired
 Parents’ preferences
 Opinion of brother or sister
 Many friends from home attend
 High quality of faculty
Rating Response Strategy

Strongly Somewhat Not at all

influential influential influential
Good academic reputation   

Enjoyable campus life   

Many friends   

High quality faculty   

Semester calendar   
Ranking

Please rank-order your top three factors from the following list
based on their influence in encouraging you to apply to Metro U.
Use 1 to indicate the most encouraging factor, 2 the next most
encouraging factor, etc.
_____ Opportunity to play collegiate sports
_____ Closeness to home
_____ Enjoyable campus life
_____ Good academic reputation
_____ High quality of faculty
Summary of Scale Types

Type Restrictions Scale Data Type

Items
Rating Scales
Simple Category • Needs mutually exclusive choices One or Nominal
Scale more
Multiple Choice • Needs mutually exclusive choices Many Nominal
Single-Response • May use exhaustive list or ‘other’
Scale

Multiple Choice • Needs mutually exclusive choices Many Nominal

Multiple-Response • Needs exhaustive list or ‘other’
Scale
(checklist)

Likert Scale • Needs definitive positive or One or Ordinal

negative statements with which to more
agree/disagree
Likert-type Scale •Needs definitive positive or One or Ordinal
negative statements with which to more
agree/disagree
Summary of Scale Types

Type Restrictions Scale Data Type

Items
Rating Scales

Numerical •Needs concepts with standardized meanings; One or Ordinal or

Scale •Needs number anchors of the scale or end-points many Interval
•Score is a measurement of graphical space
Multiple •Needs words that are opposites to anchor the Up to Ordinal
Rating List end-points on the verbal scale 10
Scale
Fixed Sum •Participant needs ability to calculate total to Two or Interval or Ratio
Scale some fixed number, often 100. more
Summary of Scale Types

Type Restrictions Scale Data Type

Items
Rating Scales

Stapel •Needs verbal labels that are operationally One or Ordinal or

Scale defined or standard. more Interval

Graphic •Needs visual images that can be interpreted as One or Ordinal

Rating positive or negative anchors more (Interval, or
Scale •Score is a measurement of graphical space Ratio)
from one anchor.
Summary of Scale Types

Type Restrictions Scale Data Type

Items
Ranking Scales
Paired • Number is controlled by participant’s Up to 10 Ordinal
Comparison stamina and interest.
Scale

Forced Ranking • Needs mutually exclusive choices. Up to 10 Ordinal or

Scale Interval

Comparative • Can use verbal or graphical scale. Up to 10 Ordinal

Scale
Internet Survey
Scale Options
Internet Survey
Scale Options
Internet Survey
Scale Options
Sources of Questions

• Handbook of Marketing • Index to International

Scales Public Opinion
• The Gallup Poll • Sourcebook of Harris
Cumulative Index National Surveys
• Measures of Personality • Marketing Scales
and Social-Psychological Handbook
Attitudes • American Social
• Measures of Political Attitudes Data
Attitudes Sourcebook
Guidelines for
Question Sequencing

Interesting
Interesting topics
topics early
early

Simple
Simple topics
topics early
early

Sensitive
Sensitive questions
questions later
later

Classification
Classification questions
questions later
later

Transition
Transition between
between topics
topics

Reference
Reference changes
changes limited
limited
Illustrating the Funnel
Approach

How do you think this country is getting along in its

relations with other countries?
How do you think we are doing in our relations with
Iran?
Do you think we ought to be dealing with Iran
differently than we are now?
(If yes) What should we be doing differently?
Some people say we should get tougher with Iran and
others think we are too tough as it is; how do you
feel about it?
Branching Question
Components of
Questionnaires
MindWriter Survey
OPEN-ENDED QUESTIONS

• Enable authentic responses;

• More time-consuming and difficult to analyze/process.
Advantages of Questionnaire method
of data collection
The merits claimed on behalf of this method are as follows:
1.There is low cost even when the universe is large and is widely
spread geographically.
2. It is free from the bias of the interviewer; answers are in
respondents’ own words.
3. Respondents have adequate time to give well thought out
answers.
4. Respondents, who are not easily approachable, can also be
reached conveniently.
5 Large samples can be made use of and thus the results can be
made more dependable and reliable.
Disadvantages of Questionnaire
method of data collection
The main demerits of this system can also be listed here:
1. Low rate of return of the duly filled in questionnaires; bias due to no-
response is often indeterminate.
2. It can be used only when respondents are educated and cooperating.
3. The control over questionnaire may be lost once it is sent.
4. There is inbuilt inflexibility because of the difficulty of amending the
approach once questionnaires have been despatched.
5. There is also the possibility of ambiguous replies or omission of replies
altogether to certain questions; interpretation of omissions is difficult.
6. It is difficult to know whether willing respondents are truly
representative.
7. This method is likely to be the slowest of all.