Chapter 1 – Introduction to online surveys
1.1 Introduction
The past 50 years are characterized with technological developments. With the rise of the
Internet, online surveying has become the dominant mode of survey data collection. In the
1980s surveys were conducted online for the first time, via email.
The advantages of online surveys compared to telephone, face-to-face, and paper-and-
pencil surveys are mainly in cost (no hiring/training interviewers, no recording equipment,
no travel) and speed (reach large numbers of people in short time, while working on other
tasks in the meanwhile).
The usefulness of online surveys depends on the Internet penetration rate of the population
(→ who can be reached).
1.2 Online surveys for quantitative and qualitative research
Surveys are more commonly used for quantitative research, but can also be used for
qualitative research. Whether to collect quantitative or qualitative data is the first decision a
researcher has to make.
Quantitative research: quantifying a research question by generating numerical data that
can be used for statistical testing
Qualitative research: gaining understating of underlying reasons, opinions, motivations.
Less structured, often exploratory.
Qualitative and quantitative are often combined → mixed-method research
1.2.1 Quantitative online research
Quantitative Web surveys can be used to answer a question about great groups of people
or to generalize its results (inferences). For large-scale research. It focuses on the ability to
perform statistical analyses. Most questions are close-ended (pick one of the answering
options). This converts answers into numbers.
1.2.2 Qualitative online research
Web surveys enable researchers to investigate Internet communities (chat rooms, mailing
lists, online games).
Qualitative studies do not require representative samples as they are not interested in the
average score but in understanding particular groups/individuals.
There are three different research methodologies for online qualitative research:
1. Passive analysis: analysis of interactions and information
2. Active analysis: analysis of interactions and information but now the researcher
interacts with the participants, often without revealing their identity
3. Web surveys: the researcher informs the participants of their identity (more ethical)
Questions in qualitative Web surveys are often open-ended (“how/why?”), free answers.
1.3 Different types of Web surveys
1.3.1 Non-probability Web surveys
→ sampling in a non-random manner, there are members that have zero chance of being
selected.
, Three types:
1. Polls as entertainment: only polls the opinion of people who happen to come across it
and wish to take part; no generalization possible
2. Unrestricted self-selected surveys: advertisements or announcements on high traffic
websites; claim to have scientific value
3. Volunteer opt-in panels: even though panel members are randomly selected, the initial
panel of volunteers consists of a self-selected group; claim to be representative of the
general population
1.2.2 Probability-based Web surveys
→ random recruitment, all members of the target population have a chance to be selected
to participate.
Five types:
1. Intercept surveys: every nth person visiting a website is invited to participate; cookies to
avoid selecting a person twice; limited but useful to measure customer satisfaction
2. List based samples: start with list of people that have access to the Internet and send
an invitation to participate in the survey by email
3. Web option in mixed-mode surveys: only participants that have a preference for
completing the survey online are invited to do so, others have different modes
4. Pre-recruited panels of Internet users: members are recruited by random digit dialling
or address-based sampling, asked whether they have internet and selected for the panel
5. Pre-recruited panels of full population: also includes non-Internet users, providing
them with alternative equipment
1.4 Total survey error
Total Survey Error: conceptual framework describing statistical error properties of sample
surveys; displays a balance of costs and errors
The ‘fitness for use’ perspective acknowledges that producers often prioritize high sample
size and response rate, while data users prioritize accessibility and usability of the data.
A design should bring a small total survey error but the costs of that design should also be
taken into account.
There are four fundamental concepts (‘cornerstones’) for a good and accurate survey:
accurate measurements, high response rate, high coverage, and proper sampling.
1.4.1 Measurement error
→ Deviation of the retrieved answers from their true value
This could be due to confusing instructions, ambiguous questions, poor visual design,
complicated terms.
As online surveys can be completed at the respondent’s time and place of choosing,
researchers cannot control the influence of environmental factors on the answers.
1.4.2 Non-response error
1. Unit non-response error: individuals not participating in the survey
2. Partial non-response error: individuals who drop out during the survey
3. Item non-response error: individuals failing to provide the answer to an individual
question or item
Non-response only leads to error when the non-respondents differ from the respondents in
, features central to the questionnaire (→ threat to validity)
Non-response is a greater problem in online surveys compared to other surveying methods
(people might accidentally click on a survey, confuse it with advertisement, etc.)
1.4.3 Coverage error
→ discrepancy between frame population and the actual target population
Often generalizing beyond the online population; demographic groups that do not have
access to the Internet differ substantially from those who do (younger, richer, more
educated) → digital divide
Coverage error does not only impact survey validity due to the digital divide but also by
Internet penetration (→ % of population that has access to the Internet)
1.4.4 Sampling error
→ the sample that is drawn from the (correct) frame population is not representative
Related to the size of the sample (too small) relative to the size of the population
1.5 Dimension differences between modes
1.5.1 Degree of interviewer involvement
Self-administered surveys (online/paper-and-pencil) are less influenced by social
desirability bias than interviewer-administered questionnaires, since the latter have more
interviewer involvement. Also, self-administered surveys save time and are less costly.
Disadvantages: low degree of interviewer involvement leads to non-observation and
measurement errors (interviewer cannot probe/motivate/assist)
1.5.2 Degree of interaction with the respondent
Related to interviewer involvement. In telephone surveys, interaction is only verbal and
therefore there is less direct contact compared to face-to-face interviews. In Web surveys,
direct contact can be completely absent. But there could also be videos/pictures/animated
avatars of the interviewer.
Humanizing cues (videos/avatar) in Web surveys do not cause greater social desirability
bias because there is no actual presence of the interviewer.
1.5.3 Degree of privacy
Web surveys offer respondents more anonymity than most other modes; reduction of the
chance that people might try to see or overhear the answers
1.5.4 Channels of communication
Telephone surveys offer the aural channel of communication, paper-and-pencil only the
visual, face-to-face surveys both. Web surveys mostly use the visual channel but more
options of presentation are possible as audio and video support can be incorporated.
1.5.5 The use of technology
Paper based surveys use no technology, the interviewer uses technology for Computer-
Assisted-Telephone-Interviewing (CATI) and Computer-Assisted-Personal-Interviewing
(CAPI). The respondent uses technology for Computer-Administered-Self-Interviewing
(CASI). For Web surveys, the use of technology is the largest, with both interviewer and
respondent using technology.
, 1.6 Internet penetration over time and place
The usability of an online survey is affected by the Internet penetration of the target
population. There is a wide variability across different countries and areas (e.g. East-West
Germany). The average Internet penetration is highest in North America (87.7%).
Internet penetration has increased rapidly over the past decade, especially in areas where it
was still low. Even though it is still relatively low in third world countries, the rapid growth
offers positive prospects for their ability to participate in Web surveys.
1.7 Web surveys on mobile devices
There is also a considerable growth in the usage of Internet on mobile smart phones. This
brings new challenges to Web survey researchers. Mobile phone screens are of small size
and are operated by either a small keyboard or a touch screen.
Benefits: possibility to use the RDD technique (solve problem that there are no random
sampling methods for Web surveys analogous to those in telephone surveys); more
freedom to choose place and time (reduce non-response rate)
