Hoorcolleges samenvattingen
Dit is een samenvatting van de hoorcolleges van Conducting a Survey 2021-2022. Dit
vak is voor het grootste deel fysiek gegeven dit jaar.
Lecture 1...................................................................................................................................................7
Introduction and specifying your research-question..................................................................7
Themes per week................................................................................................................................7
Goals of the course............................................................................................................................7
Big data vs survey data......................................................................................................................8
Characteristics of big data:...........................................................................................................8
Limitations of big data:..................................................................................................................8
So why this course?............................................................................................................................9
Your own survey.................................................................................................................................9
Assignment 1: Research plan.........................................................................................................10
Research process.........................................................................................................................10
Planning primary data collection...............................................................................................11
Research instrument: Survey.........................................................................................................11
Research project...............................................................................................................................11
Why survey research?.....................................................................................................................12
When is it an appropriate tool?................................................................................................12
GIGO...................................................................................................................................................13
Homework week 1...........................................................................................................................13
Lecture 2.................................................................................................................................................14
Introduction to question writing...................................................................................................14
Approaching writing....................................................................................................................14
How to decide which questions?.............................................................................................14
Introduction into writing questions.........................................................................................14
Pitfalls:........................................................................................................................................15
Level of measurement................................................................................................................15
Examples of different kinds of questions...............................................................................15
Question order.............................................................................................................................16
Introduction text..........................................................................................................................16
, Number of questions..................................................................................................................16
Measurement error......................................................................................................................16
Pitfalls for response categories: response effects............................................................17
Typical response effects.............................................................................................................17
Response effect depend on...................................................................................................17
Survey design choices.....................................................................................................................18
Visual design..................................................................................................................................18
Visual heuristics............................................................................................................................18
Guidelines for design choices....................................................................................................19
Design for mobile.........................................................................................................................19
Mobile technologies (AAPOR)...............................................................................................19
Analysis plan......................................................................................................................................20
Scale-analysis.....................................................................................................................................20
Scale-development...........................................................................................................................20
Classical test theory: scale evaluation.........................................................................................21
Factor analysis...................................................................................................................................21
Reliability............................................................................................................................................21
Lecture 3.................................................................................................................................................22
Total Survey Error (TSE).................................................................................................................22
Stages of answering questions (Tourangeau’s Response Model)..........................................23
Four stages of cognitive information processing after reading questions:.....................23
Interpretation............................................................................................................................24
Retrieval.....................................................................................................................................24
Judgement.................................................................................................................................24
Reporting...................................................................................................................................24
Measurement Errors........................................................................................................................25
Type of measurement error.......................................................................................................25
Survey modes....................................................................................................................................25
Recruitment...................................................................................................................................25
Mode of data collection..............................................................................................................26
Interviewer versus self-administered questionnaires......................................................26
Computer assisted interviewing...........................................................................................27
, What is best?..............................................................................................................................28
Face-to-face surveys...............................................................................................................28
Telephone surveys..................................................................................................................28
Mail surveys..............................................................................................................................28
Web surveys.............................................................................................................................28
Optimal strategy for recruitment..............................................................................................29
Tailored design.........................................................................................................................29
Mixed-Mode designs...................................................................................................................30
(Dis)advantages........................................................................................................................30
Design for mix..........................................................................................................................30
Fieldwork............................................................................................................................................30
Incentives...........................................................................................................................................30
Types of incentives......................................................................................................................31
Optimum incentive..................................................................................................................31
Survey motivation............................................................................................................................32
Reasons for refusal..................................................................................................................32
Reasons for participating.......................................................................................................32
Conclusion.........................................................................................................................................32
Lecture 4.................................................................................................................................................34
Notes on analysis plan.....................................................................................................................34
Advance letters.............................................................................................................................34
Letters of introduction................................................................................................................34
Reminders......................................................................................................................................34
Permission needed.......................................................................................................................35
Sampling.............................................................................................................................................35
Sample error versus sampling bias...........................................................................................35
Random sampling designs..........................................................................................................35
Sampling respondents.....................................................................................................................36
Sampling frame.................................................................................................................................36
Non-random sampling.................................................................................................................36
Calculating sample size...................................................................................................................37
Formula..........................................................................................................................................37
, 1. Probability level...................................................................................................................37
2. Variance.................................................................................................................................38
3. Confidence interval.............................................................................................................38
Power analysis..............................................................................................................................38
Guidelines for non-random samples........................................................................................38
Definitions AAPOR......................................................................................................................39
(non)Response rates........................................................................................................................39
Response rate 1............................................................................................................................39
Response rate 3............................................................................................................................40
Response rate 5............................................................................................................................40
Unit-nonresponse, a problem?..................................................................................................40
No problem:..............................................................................................................................40
A problem:.................................................................................................................................40
Non-response error.................................................................................................................41
What to mention about response rates..................................................................................41
Sampling plan....................................................................................................................................41
Lecture 5.................................................................................................................................................43
Straightforward Coding...................................................................................................................43
Non-straightforward....................................................................................................................43
Additional remarks...................................................................................................................43
Screening............................................................................................................................................43
Bivariate screening.......................................................................................................................44
Multivariate screening.................................................................................................................44
Potential outliers......................................................................................................................45
What to do with outliers?......................................................................................................45
Analyses..............................................................................................................................................45
Independent Samples T-Test.....................................................................................................45
More than two independent groups........................................................................................46
Histogram after ‘split file’............................................................................................................46
ANOVA through GLM................................................................................................................46
Association between variables..................................................................................................46
X2 tests..........................................................................................................................................47
Dit is een samenvatting van de hoorcolleges van Conducting a Survey 2021-2022. Dit
vak is voor het grootste deel fysiek gegeven dit jaar.
Lecture 1...................................................................................................................................................7
Introduction and specifying your research-question..................................................................7
Themes per week................................................................................................................................7
Goals of the course............................................................................................................................7
Big data vs survey data......................................................................................................................8
Characteristics of big data:...........................................................................................................8
Limitations of big data:..................................................................................................................8
So why this course?............................................................................................................................9
Your own survey.................................................................................................................................9
Assignment 1: Research plan.........................................................................................................10
Research process.........................................................................................................................10
Planning primary data collection...............................................................................................11
Research instrument: Survey.........................................................................................................11
Research project...............................................................................................................................11
Why survey research?.....................................................................................................................12
When is it an appropriate tool?................................................................................................12
GIGO...................................................................................................................................................13
Homework week 1...........................................................................................................................13
Lecture 2.................................................................................................................................................14
Introduction to question writing...................................................................................................14
Approaching writing....................................................................................................................14
How to decide which questions?.............................................................................................14
Introduction into writing questions.........................................................................................14
Pitfalls:........................................................................................................................................15
Level of measurement................................................................................................................15
Examples of different kinds of questions...............................................................................15
Question order.............................................................................................................................16
Introduction text..........................................................................................................................16
, Number of questions..................................................................................................................16
Measurement error......................................................................................................................16
Pitfalls for response categories: response effects............................................................17
Typical response effects.............................................................................................................17
Response effect depend on...................................................................................................17
Survey design choices.....................................................................................................................18
Visual design..................................................................................................................................18
Visual heuristics............................................................................................................................18
Guidelines for design choices....................................................................................................19
Design for mobile.........................................................................................................................19
Mobile technologies (AAPOR)...............................................................................................19
Analysis plan......................................................................................................................................20
Scale-analysis.....................................................................................................................................20
Scale-development...........................................................................................................................20
Classical test theory: scale evaluation.........................................................................................21
Factor analysis...................................................................................................................................21
Reliability............................................................................................................................................21
Lecture 3.................................................................................................................................................22
Total Survey Error (TSE).................................................................................................................22
Stages of answering questions (Tourangeau’s Response Model)..........................................23
Four stages of cognitive information processing after reading questions:.....................23
Interpretation............................................................................................................................24
Retrieval.....................................................................................................................................24
Judgement.................................................................................................................................24
Reporting...................................................................................................................................24
Measurement Errors........................................................................................................................25
Type of measurement error.......................................................................................................25
Survey modes....................................................................................................................................25
Recruitment...................................................................................................................................25
Mode of data collection..............................................................................................................26
Interviewer versus self-administered questionnaires......................................................26
Computer assisted interviewing...........................................................................................27
, What is best?..............................................................................................................................28
Face-to-face surveys...............................................................................................................28
Telephone surveys..................................................................................................................28
Mail surveys..............................................................................................................................28
Web surveys.............................................................................................................................28
Optimal strategy for recruitment..............................................................................................29
Tailored design.........................................................................................................................29
Mixed-Mode designs...................................................................................................................30
(Dis)advantages........................................................................................................................30
Design for mix..........................................................................................................................30
Fieldwork............................................................................................................................................30
Incentives...........................................................................................................................................30
Types of incentives......................................................................................................................31
Optimum incentive..................................................................................................................31
Survey motivation............................................................................................................................32
Reasons for refusal..................................................................................................................32
Reasons for participating.......................................................................................................32
Conclusion.........................................................................................................................................32
Lecture 4.................................................................................................................................................34
Notes on analysis plan.....................................................................................................................34
Advance letters.............................................................................................................................34
Letters of introduction................................................................................................................34
Reminders......................................................................................................................................34
Permission needed.......................................................................................................................35
Sampling.............................................................................................................................................35
Sample error versus sampling bias...........................................................................................35
Random sampling designs..........................................................................................................35
Sampling respondents.....................................................................................................................36
Sampling frame.................................................................................................................................36
Non-random sampling.................................................................................................................36
Calculating sample size...................................................................................................................37
Formula..........................................................................................................................................37
, 1. Probability level...................................................................................................................37
2. Variance.................................................................................................................................38
3. Confidence interval.............................................................................................................38
Power analysis..............................................................................................................................38
Guidelines for non-random samples........................................................................................38
Definitions AAPOR......................................................................................................................39
(non)Response rates........................................................................................................................39
Response rate 1............................................................................................................................39
Response rate 3............................................................................................................................40
Response rate 5............................................................................................................................40
Unit-nonresponse, a problem?..................................................................................................40
No problem:..............................................................................................................................40
A problem:.................................................................................................................................40
Non-response error.................................................................................................................41
What to mention about response rates..................................................................................41
Sampling plan....................................................................................................................................41
Lecture 5.................................................................................................................................................43
Straightforward Coding...................................................................................................................43
Non-straightforward....................................................................................................................43
Additional remarks...................................................................................................................43
Screening............................................................................................................................................43
Bivariate screening.......................................................................................................................44
Multivariate screening.................................................................................................................44
Potential outliers......................................................................................................................45
What to do with outliers?......................................................................................................45
Analyses..............................................................................................................................................45
Independent Samples T-Test.....................................................................................................45
More than two independent groups........................................................................................46
Histogram after ‘split file’............................................................................................................46
ANOVA through GLM................................................................................................................46
Association between variables..................................................................................................46
X2 tests..........................................................................................................................................47
