Survey Research Methods Lecture Notes – Sofie Polman
Table of Contents
Lecture 2: Introduction To Survey Research................................................................................................2
Lecture 3: Planning A Survey.......................................................................................................................5
Lecture 4: Designing A Survey – Measurement...........................................................................................8
Lecture 5: Scale Development...................................................................................................................14
Lecture 6: Designing A Survey – Questionnaire Design..............................................................................16
Lecture 7: Segmentation and Typology.....................................................................................................21
Lecture 8: Implementing A Survey............................................................................................................22
Lecture 9: Testing Relationships................................................................................................................28
Lecture 12: Wrap-Up.................................................................................................................................31
,Lecture 2: Introduction To Survey Research
What is important?
Everything related to questions
Clear questions to be able to get a proper answer
Avoid leading questions (questions that leads into a certain direction) so you won’t get unbiased
answers
Making sure that scales are valid and reliable
Have control questions to make sure that person is paying attention
Demographics, to be able to compare different groups
Social desirability is something to avoid
Everything related to sample
Good sample – sample size doesn’t say anything about the sample, you have to keep in mind the
population
Different sampling
Everything related to the analytics
Methods should be “right”
o Exploratory factor analysis
o Regression
o ANOVA
Survey length depends how many questions you can ask depends on motivation of participants
What is Survey Research?
A survey is a systematic method for gathering information with standardized questionnaires from (a sample
of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population
of which the entities are members. (Groves et al. 2009)
Systematic and Standardized – to be able to quantify the information we get
o Distinguishes itself from other techniques such as qualitative research
(A Sample of) Entities – can also be everyone in the population
o Mostly a selection of the population is included in the survey
o All entities can be included as well (e.g., employee survey)
Quantitative Descriptors
o Statistics needed to analyze the responses
A Short History of Survey Research
Census surveys were conducted as early as Ancient Egypt
o Census survey: surveys that are obliged to answer to, and you want to ask everyone in the
population
o Poll is survey you gather from part of the population what you generalize to the entire
population
, Survey as a formal research method was pioneered in the 1930-40s by sociologist Paul Lazarsfeld to
examine the effects of the radio on political opinion formation of the United States
Today, surveys provide managers with deeper insights into their customers and employees
Surveys can contribute to generalizing experimental findings to different persons and settings
Sampling procedures of survey research
Probability sampling
Selection of sampling techniques in which the chance of each case (e.g., individual) being selected from the
population is known and not zero.
→ Randomization is used instead of deliberate choice
European Social Survey with a sampling frame and exact data on
background of all individuals in the sample
Nonprobability sampling
Selection of sampling techniques in which the chance of each case (e.g.,
individual) being selected is NOT known.
→ The researcher deliberately picks the sample
Fellow students selected to participate in a survey
Time Horizon of Survey Research
Cross-Sectional Design (FOCUS)
o Opinion polls (e.g., to predict electoral results)
o Typical thesis survey (e.g., to understand consumer preferences or attitudes)
Longitudinal Design
o Repeated cross-sectional (different sample each time)
o Fixed-sample panel design (the same sample each time)
o Cohort study (follow the same people over a longer period of time, usually after an event;
e.g., people graduating MSc Marketing in 2010 and follow their lives over a 10-year time
period)
Administration methods of survey research
Interview surveys
o Personal surveys
o Telephone surveys
Self-completion surveys
o Written surveys
o Online surveys
depends on a lot of factors (time, resources) which is best/most suitable
Why so Popular? And What are the Challenges?
Main Pros
Ease: Questionnaires are relatively easy to administer to large number of respondents
Simplicity: coding, analysis, and interpretation of data is relatively simple
Reliability: Standardization reduces variability in the answers
Challenges
Asking the right questions is not easy
Properly wording questions is not easy
Getting the right sample (sampling) is not easy
Motivating respondents to respond (candidly) is not easy
Publishing survey research is not easy because of certain biases about the information you can get from
surveys.
, Total Survey Error = Measurement and Representation
1. Measurement do you actually measure what you want to
measure?
What is the survey about or, more specifically, what does the
survey measures, and do the observed responses actually measure
what the researcher is interested in?
2. Representation (& sample when collecting data)
Whom does the survey describe, and if the survey does not involve
self-reports, who provides the desired data?
3. Analysis and Reporting
What can we learn from the survey?
Errors that invalidate survey results may arise in all areas.
There are also additional sources of error and bias...
Measurement in the Survey (Groves et al. 2004)
Construct: (Latent) Variable and its relationships
o Covid infections and age
Measurement: Form in which (latent) variable is assessed
o Question: “Did you have Covid in the past six months?”
Response: Response provided
o Age (Year of birth)
o Open answer: Yes or No
Edited Response
o Year of birth: 1483, 27, 201, 2200?
o Edited answer: “perhaps” is not valid?
Population: Set of units under study
o Population: All adults in the Netherlands
Sampling Frame
o Set of eligible units to survey
o Sampling Frame: Dutch phone no. directory
Sample: Set of units to survey
o Sample: 10,000 Dutch phone no.
Respondents
o Set of successfully surveyed units
o Respondents: 1,214 AI-powered interviews
o 31 percent male, 69 percent female
Adjusted Respondents
o Final set of respondents for analysis
o Adjusted respondents: Weighted sample
A survey cannot show you causality – and sometimes does not take all factors into account (you have to
control for other factors to get to quasi-causality)
Additional survey biases: trusting and publishing results
Table of Contents
Lecture 2: Introduction To Survey Research................................................................................................2
Lecture 3: Planning A Survey.......................................................................................................................5
Lecture 4: Designing A Survey – Measurement...........................................................................................8
Lecture 5: Scale Development...................................................................................................................14
Lecture 6: Designing A Survey – Questionnaire Design..............................................................................16
Lecture 7: Segmentation and Typology.....................................................................................................21
Lecture 8: Implementing A Survey............................................................................................................22
Lecture 9: Testing Relationships................................................................................................................28
Lecture 12: Wrap-Up.................................................................................................................................31
,Lecture 2: Introduction To Survey Research
What is important?
Everything related to questions
Clear questions to be able to get a proper answer
Avoid leading questions (questions that leads into a certain direction) so you won’t get unbiased
answers
Making sure that scales are valid and reliable
Have control questions to make sure that person is paying attention
Demographics, to be able to compare different groups
Social desirability is something to avoid
Everything related to sample
Good sample – sample size doesn’t say anything about the sample, you have to keep in mind the
population
Different sampling
Everything related to the analytics
Methods should be “right”
o Exploratory factor analysis
o Regression
o ANOVA
Survey length depends how many questions you can ask depends on motivation of participants
What is Survey Research?
A survey is a systematic method for gathering information with standardized questionnaires from (a sample
of) entities for the purpose of constructing quantitative descriptors of the attributes of the larger population
of which the entities are members. (Groves et al. 2009)
Systematic and Standardized – to be able to quantify the information we get
o Distinguishes itself from other techniques such as qualitative research
(A Sample of) Entities – can also be everyone in the population
o Mostly a selection of the population is included in the survey
o All entities can be included as well (e.g., employee survey)
Quantitative Descriptors
o Statistics needed to analyze the responses
A Short History of Survey Research
Census surveys were conducted as early as Ancient Egypt
o Census survey: surveys that are obliged to answer to, and you want to ask everyone in the
population
o Poll is survey you gather from part of the population what you generalize to the entire
population
, Survey as a formal research method was pioneered in the 1930-40s by sociologist Paul Lazarsfeld to
examine the effects of the radio on political opinion formation of the United States
Today, surveys provide managers with deeper insights into their customers and employees
Surveys can contribute to generalizing experimental findings to different persons and settings
Sampling procedures of survey research
Probability sampling
Selection of sampling techniques in which the chance of each case (e.g., individual) being selected from the
population is known and not zero.
→ Randomization is used instead of deliberate choice
European Social Survey with a sampling frame and exact data on
background of all individuals in the sample
Nonprobability sampling
Selection of sampling techniques in which the chance of each case (e.g.,
individual) being selected is NOT known.
→ The researcher deliberately picks the sample
Fellow students selected to participate in a survey
Time Horizon of Survey Research
Cross-Sectional Design (FOCUS)
o Opinion polls (e.g., to predict electoral results)
o Typical thesis survey (e.g., to understand consumer preferences or attitudes)
Longitudinal Design
o Repeated cross-sectional (different sample each time)
o Fixed-sample panel design (the same sample each time)
o Cohort study (follow the same people over a longer period of time, usually after an event;
e.g., people graduating MSc Marketing in 2010 and follow their lives over a 10-year time
period)
Administration methods of survey research
Interview surveys
o Personal surveys
o Telephone surveys
Self-completion surveys
o Written surveys
o Online surveys
depends on a lot of factors (time, resources) which is best/most suitable
Why so Popular? And What are the Challenges?
Main Pros
Ease: Questionnaires are relatively easy to administer to large number of respondents
Simplicity: coding, analysis, and interpretation of data is relatively simple
Reliability: Standardization reduces variability in the answers
Challenges
Asking the right questions is not easy
Properly wording questions is not easy
Getting the right sample (sampling) is not easy
Motivating respondents to respond (candidly) is not easy
Publishing survey research is not easy because of certain biases about the information you can get from
surveys.
, Total Survey Error = Measurement and Representation
1. Measurement do you actually measure what you want to
measure?
What is the survey about or, more specifically, what does the
survey measures, and do the observed responses actually measure
what the researcher is interested in?
2. Representation (& sample when collecting data)
Whom does the survey describe, and if the survey does not involve
self-reports, who provides the desired data?
3. Analysis and Reporting
What can we learn from the survey?
Errors that invalidate survey results may arise in all areas.
There are also additional sources of error and bias...
Measurement in the Survey (Groves et al. 2004)
Construct: (Latent) Variable and its relationships
o Covid infections and age
Measurement: Form in which (latent) variable is assessed
o Question: “Did you have Covid in the past six months?”
Response: Response provided
o Age (Year of birth)
o Open answer: Yes or No
Edited Response
o Year of birth: 1483, 27, 201, 2200?
o Edited answer: “perhaps” is not valid?
Population: Set of units under study
o Population: All adults in the Netherlands
Sampling Frame
o Set of eligible units to survey
o Sampling Frame: Dutch phone no. directory
Sample: Set of units to survey
o Sample: 10,000 Dutch phone no.
Respondents
o Set of successfully surveyed units
o Respondents: 1,214 AI-powered interviews
o 31 percent male, 69 percent female
Adjusted Respondents
o Final set of respondents for analysis
o Adjusted respondents: Weighted sample
A survey cannot show you causality – and sometimes does not take all factors into account (you have to
control for other factors to get to quasi-causality)
Additional survey biases: trusting and publishing results
