Survey Research, Steps in Survey

Research, Communication
Methods in Survey Research

Chapter 10
 Steps in Survey Research
1. State the objectives of the survey
2. Define the target population
3. Define the data to be collected
4. Define the required precision and accuracy
5. Define the measurement `instrument'
6. Define the sample frame, sample size and
sampling method
7. Select the sample
8. Collect the data
9. Data analysis
10. Results
2
 Steps in Survey Research
1. State the objectives of the survey.
You have to define specifically the problem you are trying to solve.
If you cannot state the objectives of the survey you are unlikely to
generate useable results. You have to be able to formulate
something quite detailed, perhaps organized around a clear
statement of a testable hypothesis. Clarifying the aims of the survey
is critical to its ultimate success.

3
 Steps in Survey Research
2. Define the target population.
Defining the target population can be relatively simple, especially
for finite populations, however, it may be more difficult to define
what constitutes 'natural' membership of the population; In that
case, arbitrary decisions have to be made.

The process of defining the population is quite different when

dealing with continuous (rather than discrete) phenomena. As you
will see, it is still possible to define a sample size even if you don't
know the proportion of the population that the sample represents.

4
 Steps in Survey Research
3. Define the data to be collected.
What new information do you need to solve the problem?
Hint: prepare hypothetical tables of results. They help us to separate
“need to know” than “nice to know”.

Focus groups can help to find out which questions to ask in a

survey. But they can not substitute surveys.

5
 Steps in Survey Research
4. Define the required precision and accuracy
The most subjective stage is defining the precision with which the
data should be collected. Strictly speaking, the precision can only be
correctly estimated if we conduct a census. The precision provided
by a sample survey is an estimate the 'tightness' of the range of
estimates of the population characteristics provided by various
samples.

When we estimate a population value from a sample we can only

work out how accurate the sample estimate is if we actually know
the correct value - which we rarely do - but we can estimate the
'likely' accuracy. We need to design and select the sample in such a
way that we obtain results that have acceptable precision and
accuracy

6
 Steps in Survey Research
5. Define the measurement `instrument‘.
The measurement instrument is the method - interview,
observation, questionnaire - by which the survey data is
generated.

To produce useful information the ideas that motivated

the survey must be translated into good questions.

7
 Steps in Survey Research
6. Define the sample frame, sample size and sampling
method.
The sample frame is the list of people ('objects' for
inanimate populations) that make up the target
population; It is a list of the individuals who meet the
'requirements' to be a member of that population.
The sample is selected from the sample frame by
specifying the sample size (either as a finite number, or
as a proportion of the population).
The sampling method is the process by which we choose
the members of the sample.

8
 Steps in Survey Research
7. Select the sample.
• The sample is selected, using the sample method
defined, from the sample frame by specifying the sample
size.

• The process of generating a sample requires several critical

decisions to be made. Mistakes at this stage will compromise -
and possibly invalidate - the entire survey. These decisions are
concerned with the sample frame, the sample size, and the
sampling method.

9
•
Types of Errors.
Coverage error occurs when the list (or frame) from which a
sample is drawn does not include all elements of the population
that researchers wish to study.
• Sampling error occurs when researchers survey only a subset or
sample of all people in the population instead of conducting a
census.
• Measurement error occurs when a respondent’s answer to a
giving question is inaccurate, imprecise, or can not be compared in
any useful way to other respondent’s answers.
• Nonresponse error occurs when a significant number of people in
the survey sample do not respond to the questionnaire and are
different from those who do in a way that is important to the study.
• 1. Total nonresponse. A questionnaire is missing (not returned).
• 2. Item nonresponse. A specific item of the questionnaire is missing (not
answered).

10
 Steps in Survey Research
8. Collect the data.
Apply the instrument to collect the information.

There are different models to collect the data.

• Telephone survey
• Direct administration to a group
• Personal interview
• Mail
• Internet survey and e-mail

11
 Steps in Survey Research
9. Data analysis
• Clean the questionnaires.
• Code the questionnaires.
– Close-ended questions.
– Open-ended questions.
– Partially close-ended questions.
– Missing data.
• Decide which statistics are most useful to you.
• Interpretation. Look for results that matter.

12
 Steps in Survey Research
10. Results
• Abstract or executive summary.
• Problem statement.
• Methods and procedures.
• Error structure.
• Findings.
• Implications.
• Appendices.

13
Broad Classification of Communication
Methods in Survey Research
a) Interviewer-administered survey
methods
1. Personal Interviews.
2. Telephone Interviews.
b) (Respondent) Self-administered survey
methods.
1. Paper-based
2. Electronic
c) Questionnaires
d) Mixed-mode surveys.
14
 PERSONAL INTERVIEWS
Personal Interview – A form of direct communication
in which an interviewer asks respondents questions
face-to-face.
a) Door-to-Door Interviews – Interviews conducted at
respondents’ doorsteps (in their homes)
b) Mall Intercept Interviews (Shopping Center
Sampling) – Interviews conducted by intercepting
respondents at a central location in a shopping mall
c) Pre-arranged Meetings – Interviewing respondents
at other mutually agreed upon places

15
Advantages of Personal Interviews
1. Opportunity for feedback.
2. Opportunity to probe complex questions.
3. Willingness of respondents to spend longer time
on interview.
4. Reduction in rate of item nonresponse
(completeness of questionnaire).
5. Possibility of using visual aids & props.
6. High response rate / willingness of respondents
to participate.

16
Disadvantages of Personal Interviews
1. Interviewer influence.
2. Respondent cannot be anonymous.
3. There may be a need for several callbacks
(attempts to re-contact individuals who have
been selected for a sample but who were not
available during the earlier visit)
4. High cost.

17
 TELEPHONE INTERVIEWS
• Interviewer-administered interviews that are
conducted over the telephone.
• Telephone interviews are the most widely
used method in commercial survey research

Telephone Interviewing Systems

a) Central Location Interviewing.
b) Computer-Assisted Telephone Interview.
c) Computerized Voice-Activated Telephone
Interviewing.
18
 Telephone Interviewing Systems
1. Central Location Interviewing:
Conducting telephone interviews from a single
central location from which all interviewers work
 Usually through WATS (Wide Area
Telecommunications Service) contracts that provide
fixed long-distance rates, and allow unlimited calls
within a specific geographic area
 Allows Supervision and control.

19
 Telephone Interviewing Systems
2. Computer-Assisted Telephone Interviewing (CATI):
Telephone interviewing system in which the
interviewer reads questions from a computer screen
and enters the answers directly into the computer
• Usually incorporate telephone management
systems that handle:
a) Phone number selection.
b) Automatic dialing.
c) Sample selection.
d) Automatic callback scheduling.
e) Reporting on number of completed interviews.

20
 Telephone Interviewing Systems
3. Computerized Voice-Activated Telephone
Interviews
• Computer assumes responsibility for all
aspects of the interview administration
• It dials the respondent, asks the questions,
and records the responses.
– Used mainly for short, simple questionnaires.

21
Advantages of Telephone Interviews
1. Relatively high speed of data collection.
2. Inexpensive compared to personal
interviews.
3. Better respondent anonymity than personal
interviews.
4. Relatively higher respondent cooperation 
lower nonresponse compared to personal
interviews.

22
 Disadvantages of Telephone
Interviews
1. Problems in getting representative samples;
unlisted phone numbers; random digit dialing as
solution.
2. Problem of answering machines & faxes.
3. Need for callbacks.
4. Respondent can easily hang up.
5. Inability to use visual aids.
6. Need for shorter forms of questioning.

23
[RESPONDENT] SELF-ADMINISTERED SURVEYS
• Surveys in which respondents take the
responsibility for reading and answering the
questions.
• Present a special challenge in questionnaire
design because self-administered surveys rely
on the efficiency of the written word rather than
the skill of the interviewer.
• Self-administered questionnaires can be
classified into paper-based (printed) or
electronic (non-printed)

24
EXHIBIT 9.1 Self-Administered Questionnaires Can Be Either Printed or Electronic

25
Paper (Printed) Self-Administered Questionnaires

Distribution Methods
1. Mail (or Postal) Surveys – using regular mail
system (now usually called “Snail Mail Surveys”).
2. Drop-off / Pick-up method.
3. Fax surveys
4. Inserts – e.g. in product packages, magazines, etc.
5. Direct distribution to customers – e.g. in
restaurants, airplanes, banks.
6. Warranty / Owner registration cards.

26
Advantages of Postal and Other Printed
Self-Administered Surveys
1. Geographic flexibility.
2. Lower cost.
3. Respondent convenience
– Respondent can take time to think about
response.
– Respondent has chance to check records for
information.
4. Respondent anonymity.

27
Disadvantages of Postal and Other Printed
Self-Administered Surveys
1. Absence of interviewer means:
– Loss of control over the data collection.
– Respondent can misinterpret questions.
– Visual props cannot be used.
2. Questions have to be standardized.
3. Takes long time to receive response.
4. Questionnaire has to be short.
5. Low response rates.
6. No assurance that intended subject actually
completed the questionnaire.
28
RESPONSE RATE ISSUES IN (POSTAL) SURVEYS

• Response rate:
– Number of completed and returned
questionnaires divided by number of eligible
people contacted.

29
Methods for Increasing Response Rate
1. Cover letter explaining:
• Study importance.
• Request for help.
• Importance of respondent.
• Sampling method.
• Description of reward.
• Promise of confidentiality.
• Etc.
2. Rewards (monetary & non-monetary).
3. Advance notification.
4. Follow-ups.
5. Prestige of sponsoring institution.
6. Color of questionnaire paper.
7. Type of postage
8. Interesting questions.
30
 Effects of Different Methods for Increasing
Response Rates in Mail Surveys
Increase in response rate
over control group
Method
Prior telephone calls 19%
Monetary incentives:
$ 0.10 17%
$ 0.25 19%
$1 26%
$ 0.20 15%
31
 Effects of Different Methods for Increasing
Response Rates in Mail Surveys
Increase in response rate
over control group
Method
Non-monetary:
Pen 12%
Pocket knife 15%
Stamp vs. business reply 7%
Anonymity (in-company) 20%
Anonymity (external) 10%
Follow-ups 12%
32
Electronic Self-Administered Questionnaires
Distribution Methods
1. E-mail
a) Text
b) HTML
c) File attachments
2. Internet
a) Bulletin boards
b) Web HTML
c) Downloadable surveys
3. Kiosk Interactive Surveys
4. On-site computers
5. Disk-by-mail (CD-ROM by mail)

33
Advantages of E-mail Questionnaires

1. Speed of distribution
2. Lower distribution costs
3. Faster turnaround (response) time
4. Greater flexibility
5. Less handling of paper

34
Disadvantages of E-mail Questionnaires

1. Lack of security (Eavesdropping by

administrators)
2. Lack of anonymity
3. Differences in capabilities of
respondent’s computers and e-mail
software
4. Difficulties for e-mail novices

35
 Advantages of Internet Surveys
1. Speed
2. Cost-effectiveness
– Eliminate cost of paper, postage, and data entry
– Larger samples can be obtained
3. Visual appeal and interactivity
– More sophisticated lines of questioning
– Can use color, sound, animation, and visual materials
4. Accurate real-time data capture
5. Easy callback capabilities
6. Personalized and flexible questioning
7. Respondent anonymity

36
Disadvantages of Internet Surveys

1. Non-widespread use of internet in the

general population
2. Differences in capabilities of respondent’s
computers and connection speeds
3. Differences in computer skills
4. Lack of security for personal information

37
Advantages and Disadvantages of Typical Survey
Methods

38
Advantages and Disadvantages of Typical Survey
Methods

39
ISSUES IN CHOICE OF COMMUNICATION METHOD

• Is the assistance of an interviewer necessary?

• Are respondents interested in the issues being
investigated?
• Will cooperation be easily attained?
• How quickly is the information needed?
• Will the study require a long and complex
questionnaire?
• How large is the budget?

40
 Ethical Issues in Survey Research
• The American Marketing Association’s code
of ethics expresses researchers’ obligation to:
– Protect the public from misrepresentation and
exploitation under the guise of marketing
research
– Protect respondents’ right to privacy
– Avoid the use of deception
– Inform respondents about the purpose of the
research
– Maintain confidentiality and honesty in collecting
data
– Maintain objectivity in reporting data

41