Summary Research Methodology
and Descriptive Statistics (test 2)
Pre-master Communication Science University of Twente 2020
Britt Heuvel
,Content
This course introduces the basic principles of empirical research in the social sciences. Both
the role of research in the context of academic science (i.e. description and testing of
theories) and research in the context of problem solving and design will be discussed. This
course will be assessed in two tests with Multiple Choice questions, a SPSS test and the
assignments. This summary focusses on the first test. Test 1 consists out of unit 1 t/m 11, 13,
23 and 24, the accompanying articles and some pages from the book The Practice of Social
Research by Earl Babbie. All these parts are present in the summary.
Summary Research Methodology and Descriptive Statistics Part 1
Chapter 1 What is empirical research? (unit 1)
Chapter 2 What are clear research questions? (unit 2)
Chapter 3 What are data? (unit 3)
Chapter 4 Handling data with software (unit 4)
Chapter 5 Conceptualizing constructs (unit 5)
Chapter 6 Operationalization and data collection (unit 6)
Chapter 7 Measuring constructs using content analysis (unit 7)
Chapter 8 Two aspects of data quality (unit 8)
Chapter 9 Displaying univariate data (unit 9)
Chapter 10 Summarizing ratio variables (unit 10)
Chapter 11 Distributions and Z-scores (unit 11)
Chapter 12 Normal distribution (unit 23)
Chapter 13 Visualizing and analyzing bivariate relationships (unit 13)
Chapter 14 Describing the association between two variables (unit 24)
Summary Research Methodology and Descriptive Statistics Part 2
Chapter 15 Causality and bivariate causal hypotheses (Unit 12)
Chapter 16 Research designs for testing causal hypotheses (Unit 15)
Chapter 17 Causality and the effect of third variables (Unit 14)
Chapter 18 Elaboration: analyzing multi-variate relationships using tables I (Unit 16)
Chapter 19 Elaboration: analyzing multi-variate relationships using tables II (Unit 17&18)
Chapter 20 Sampling (Unit 19)
Chapter 21 First steps towards inference: certainty about means (Unit 20)
Chapter 22 First steps towards inference: effects and significance (Unit 21)
Chapter 23 Research Ethics (Unit 22)
Chapter 24 Formulas
Summary Research Methodology and Descriptive Statistics SPSS
General information
2
,Summary Research Methodology
and Descriptive Statistics Part 1
3
,Chapter 1 What is empirical research? (unit 1)
What is this chapter about? Where does scientific knowledge come from? The answer is: empirical research. This
chapter is about what kind of questions can be answered with empirical research, where these questions come
from and which steps need to be taken to answer the questions.
1.1 What do we mean by research?
We know things and have knowledge because we do research (this course= about empirical research)
There are different types of research questions and they require different ways of finding the answer:
1 Normative Questions à are about what is allowed or what is good
2 Conceptual Questions à are about the proper/useful/efficient meaning of words
3 Empirical Questions à are about ‘truth’ and ‘observations’.
• Two main types of empirical research questions:
1. Descriptive à are about describing facts, either at one point in time or over time
2. Explanatory (or Causal) à are about explaining the causes for something
3. + Predictive Questions (not Descriptive and not Explanatory) à are about things that will happen in
the (still unknown) future
What is Empirical Research? à Systematically (excluding the possibility that other answers are better than the
answers we give) answering empirical (things we can observe by using our senses) questions (using
observations). à Empirical Research Questions à are answered by thinking and observing at the same time.
Where do Empirical Research Questions come from?
A. Science: Follow up existing theories and puzzles
B. Decision Making: More practical, we try to solve problems
How to answer Empirical Research Questions?
à We follow a procedure: Wheel of Science à
1.) Thinking = Procedure
2.) Planning = Research Design
3.) Observing = Data Collection
4.) Analyzing = Data Analysis
1.2 Empirical Research Questions in the context of Decision Making and
Design
Empirical Research Questions are often asked in the context of Decision
Making and Design (‘How to’ questions) à These ‘How to’ questions can
be ‘broken up’ into Empirical Research Questions à they are Empirical
Questions, but they look different.
What is Decision Making? à Cycle à
Decision Making gives rise to Empirical Research Questions à How big is
the problem? Etc. à All these questions are simple Empirical Questions;
they are descriptive or explanatory.
1.3 Article Decision making and empirical research – Henk van der Kolk
The text may suggest that actual decision-making is (a) done in ‘steps’ and (b) is largely a one cycle activity.
‘Real’ decision-making is often skipping a few steps. This saves time, but it is likely this may sometimes result in
bad decisions. In addition, a lot of decision-making is better conceptualized as a large number of cycles.
1.4 Conspiracy Theories and Confirmation Bias
A Conspiracy Theory = a special type of argumentation. The start is often simple and sometimes plausible à
The attitude people have towards these theories is to prove that they are wright: Looking only for confirmations,
disregarding potential falsifications, disregarding plausible alternative explanations & contradicting counter
arguments using the idea of conspiracies.
4
,The definition of Confirmation Bias à Search for, analyze and recall information in a way that confirms pre-
existing beliefs, while giving disproportionately less consideration to alternative interpretations. You constantly
look for information that confirms what you already believe. This affects everything we do; it affects what we
think is true. (Finding the answers that we ‘want to’ find that confirm pre-existing beliefs or favored hypotheses
by ignoring evidence and/or avoiding critical evaluation).
à Three types of Confirmation Bias
• Bias in acquisition of Information (only info that confirm our beliefs)
• Bias in Reasoning (those that are in line with what we thought)
• Bias in remembering Conclusions (remember only the conclusions that are in line)
Why is Confirmation Bias so strong? à Consequences of Confirmation Bias à Mistakes in
• Limitations (in humans, relying on knowledge (we think we know things, but they are
heuristics) wrong) and Bad Decisions (example = group
• Wishful thinking (we hope some things are thinking).
true)
• Consistency (We believe one thing so The definition of Agreement Reality à The things
therefore something else is also true) we think are true, the dominant idea.
How to solve/avoid Confirmation Bias? à Follow the rules of scientific inquiry (do proper research, intelligence is
not enough), avoid Confirmation Bias by using the procedure of the Wheel of Science:
• Make clear predictions/ statements (Theory)
• Make a plan for testing these statements (Research Design)
• Collect and analyze data according the plan you had (Data Collection)
• Take your conclusions seriously (Data Analysis)
Wheel of Science à Not a logistical sequence of steps: anything goes (data,
theory). What matters is that you defend your conclusions in a logical sequence
of steps:
1. Deduction: The process of starting with theory and then thinking how
we can test the theory
2. Induction: The process of starting with data and then trying to arrive at
conclusions on the basis of data
1.5 The Practice of Social Research - Earl Babbie (Ch 1 p. 5-14)
The subject of this book is how we find out about social reality. Inquiry is a natural human activity. Much of
ordinary human inquiry seeks to explain events and predict future events. When we understand through direct
experience, we make observations and seek patterns or regularities in what we observe. Much of what we know,
we know by agreement rather than by experience. In particular, two important sources of agreed-on knowledge
are tradition and authority. However, these useful sources of knowledge can also lead us astray.
Science seeks to protect against the mistakes we make in day-to-day inquiry. Whereas we often observe
inaccurately, researchers seek to avoid such errors by making observation a careful and deliberate activity. We
sometimes jump to general conclusions on the basis of only a few observations, so scientists seek to avoid
overgeneralization. They do this by committing themselves to a sufficient number of observations and by
replicating studies. In everyday life we sometimes reason illogically. Researchers seek to avoid illogical reasoning
by being as careful and deliberate in their reasoning as in their observations. Moreover, the public nature of
science means that others are always there to challenge faulty reasoning.
Social theory attempts to discuss and explain what is, not what should be. Theory should not be confused with
philosophy or belief. Social science looks for regularities in social life. Social scientists are interested in explaining
human aggregates, not individuals. Theories are written in the language of variables. A variable is a logical set of
attributes. An attribute is a characteristic. Sex, for example, is a variable made up of the attributes male
and female. So is gender when those attributes refer to social rather than biological distinctions. In causal
explanation, the presumed cause is the independent variable, and the affected variable is the dependent
variable.
5
,Chapter 2 What are Clear Research Questions? (unit 2)
What is this chapter about? Every study starts with a research question. This chapter is about the different kinds
of research questions and the elements in research questions.
2.1 From topic to Research Questions
The Research Question (RQ) (general idea of what you hope to learn from your research) is the starting point for
a research process à that process will give you an answer. Your starting point is a topic (too broad) à this is not
a research question à you need a relevant and answerable RQ. You first start with: Why are you interested in the
topic? à Science vs. Decision Making. The first thing you do is finding books & articles: Where do you find
articles? à Google scholar/ library à How to read an article? à 1. Always read the abstract first and 2. Look at
the general structure of empirical articles (intro + RQ, theory, design, data analysis, conclusion) à *Note: only
refer to articles you have actually read à no indirect citations. A literature review is not a summary of a set
articles and books à we have to use the literature to inform our own work à we hope to find a theory.
The topic is not a research question à it maybe suggests that there is a relationship à before we start your
empirical research à Ask Conceptual Questions à We need to clarify words from the topic before we proceed.
Also think about why we expect a relationship à This is called: theory.
2.2 Structure of a Research Question
Structuring a Research Question à because of background knowledge à we have some conceptual clarification
When formulating a research question à we have to take a few things into account to have a clear question:
• Context: Science or Decision Making? (what is the context you’re working in?)
o Two contexts of scientific research questions (but they overlap)
o Practical problems (political, social problems etc.)
o Theoretical Questions (puzzles, existing research etc.)
• Normative, Conceptual or Empirical? (what type of question are you asking?)
o If Empirical, is it Descriptive or Explanatory?
• What are the Variables, Units and the Setting? (Without those questions become vague)
• What is already known? More Inductive or Deductive? (Is it answerable?)
2.3 Types of Research Questions
Different types of research questions require different kinds of procedures à why we have to distinguish them:
1. Normative (what should be the case, is it justifiable?)
o Often starts with ‘Should we …?’ or ‘Is it justifiable …?’
o Not asking for a legal fact and cannot be answered using observations only
2. Conceptual (what does it mean?)
o Often starts with ‘What is …?’
o Often just based on agreement, cannot be answered using observations
3. Empirical (what is or will be and why?)
o Answered by using observations à only clear if they refer to meaningful:
• Units of analysis à the object the RQ is about
o Identifying units if variable is known à ask: ‘What or who is characterized by this
variable?’ à This is the unit (person, city, company, year, etc.)
• Variables à the possible characteristics of these units
o Identifying variables if the unit is known à ask: ‘What characteristics does the unit
have?’ à This is the variable (unemployment, income, quality, etc.)
• Setting à Time and place or context
o Country? Date?
• Two types of Empirical Research Questions:
a) Descriptive Questions (about description)
o Some non-causal has two variables, most have one
b) Explanatory Questions (about causes and effects)
o Many have two variables, however not all are causal
o Not all refer explicitly to two variables
6
,2.4 The Practice of Social Research - Earl Babbie (Ch1 p. 14-28 + Ch4 p. 89-93)
Three major purposes of social research are exploration, description, and explanation. Studies may aim to serve
more than one of these purposes. Whereas idiographic explanations present specific cases fully, nomothetic
explanations present a generalized understanding of many cases. Inductive theories reason from specific
observations to general patterns. Deductive theories start from general statements and predict specific
observations. The underlying logic of traditional science implicitly suggests a deterministic cause-and-effect
model in which individuals have no choice, although researchers do not say, nor necessarily believe, that. Some
researchers are intent on focusing attention on the “agency” by which the subjects of study are active, choice-
making agents. The issue of free will versus determinism is an old one in philosophy, and people exhibit
conflicting orientations in their daily behavior, sometimes proclaiming their freedom and other times denying it.
Quantitative data are numerical; qualitative data are not. Both types of data are useful for different research
purposes. Research projects often begin with the preparation of a research proposal, describing the purpose
and methods of the proposed study.
Any research design requires researchers to specify as clearly as possible what they want to find out and then
determine the best way to do it. The principal purposes of social research include exploration, description, and
explanation. Research studies often combine more than one purpose. Exploration is the attempt to develop an
initial, rough understanding of some phenomenon. Description is the precise measurement and reporting of the
characteristics of some population or phenomenon under study. Explanation is the discovery and reporting of
relationships among different aspects of the phenomenon under study. Whereas descriptive studies answer the
question “What is so?” explanatory ones tend to answer the question “Why?”. Idiographic explanation seeks an
exhaustive understanding of the causes producing events and situations in a single or limited number of cases.
Pay attention to the explanations offered by the people living the social processes you are studying.
Comparisons with similar situations, either in different places or at different times in the same place, can be
insightful. Both idiographic and nomothetic models of explanation rest on the idea of causation. The idiographic
model aims at a complete under-standing of a particular phenomenon, using all relevant causal factors. The
nomothetic model aims at a general understanding (not necessarily complete) of a class of phenomena, using a
small number of relevant causal factors.
7
,Chapter 3 What are Data? (unit 3)
What is this chapter about? To answer a research question, you need to analyze data. But what are data? This
chapter is about the data matrix and that it consists of variables and units.
3.1 Units and variables
All types of Research Questions (normative, conceptual, empirical) have their own methods à focus on Empirical
o A (Empirical) Research Question has à Units of Analysis (the thing we want to analyze) and/or Units of
Observation (units in the data, the thing we observe) à A research project can also have many Nested
Units (employees à departments à plants à companies)
o All Units have characteristics à Variables à a complete and mutually exclusive set of values/ attributes
o Values/ attributes are two words for more or less the same thing
§ Two options (gender)= attributes (non-numerical)
§ Many options (age) = values (numerical)
Two Rules: Exhaustive and Mutually Exclusive:
• Exhaustive (Complete): Every unit should have a value of the variable (exhaustive set of categories)
o Every individual should at least have one of the values of a variable.
o Example: gender = 2 values à every individual you study is either male or female
• Mutually Exclusive: Every unit should have only one value of a variable
o Mutually exclusive categories
o Example: when you say that gender has 2 values à an individual you study can’t be both
3.2 Units of Analysis and Units of Observations
We have to sample and measure Units to get data:
• Units of Analysis and Units of Observation are often
of the same type à Process of Sampling à
• But Units of Analysis and Units of Observation may
be different à sometimes we select Informants à to
say something about the aggregates they are part of
or are familiar with.
o Example: interested in companies (Units of
Analysis) and they are observed by the CEO
(Units of Observation) of that company
• Units of observation are the units we get data from
• Units of analysis are the units we’re interested in
• Make sure the two are linked (CEO knows enough about company)
Research Questions about Aggregates (a whole formed by combining several separate elements):
• Units of Analysis and Units of Observation are often different
o Example: Married couples (Units of Analysis) à we collect data from one partner and the other
partner à Individuals (Units of Observation)
• Ecological Fallacy = Drawing conclusions about lower level units, only on the basis of aggregate data
o Not necessarily correct à Sometimes you can draw two types of conclusions that show that we
have to be very careful when interpretating this kind of data
3.3 Measurement levels of variables
Type of Variable Methods Terminology Explanation SPSS Terms R Terms
Nominal Dichotomy Only 2 attributes Nominal Factor
Variables Nominal More than 2 values (not ordered)
Ordinal If there is a specific order* Ordinal Numeric
Scale Variables Interval Equally spaced values* Scale or integer
Ratio Is there a clear zero point (easy x2) *
Text We can store data as text String Character
* Sometimes the data are stored as integers à this is a variable with values 1, 2, 3, 4, 5 but never 1.5
8
,3.4 Variable as constructions
We can conceptualize/ construct the variable in different ways à which conceptualization is best? Depends on
what you expect. Example age:
• Age as a Ratio Scale à measured or defined in years (months, days, minutes à depends on question)
• Age as an Ordinal Scale à measured in four categories (1. 0-18, 2. 19-35, 3. 36-64, 4. 65-older)
o We don’t know if someone’s twice as old
• Age as a Dichotomy à measured in two groups (1. 0-18 and 2. Over 18) (depends on theory)
3.5 Broad format Data Matrix
How do we arrive from a question to a Data Matrix? à Research
Question à We have specified Units and Theoretical Variables à we
need a sample of the Units and we have to operationalize (example=
Survey) the Variables to make them measurable à these things give us
Data à Data we have are stored in a Data Matrix
A Data Matrix with only numbers isn’t helpful à we don’t know what the numbers mean à so we have to label
the variables (in order to read the data matrix)à we have to attach these labels in order to make in readable.
• A codebook à describes the meaning of variables and values used in the data matrix
• A variable name à should be short (WORK, WORKYN, V1)
o Sometimes separate variables depending on the question (V1a, V1b, V1c etc.)
• A variable label à should be clear: meaning of the variable (V1= Is respondent working or not?)
• A value label à meaning of the value (0= not work, 1= work, 999= no answer)
o Ratio variables are most often not labelled in a data matrix (weight, distance)
Example Broad Data Matrix:
The Data summarizes what the Units say about the topics from the
survey à That’s how the data matrix will be constructed.
Start with a survey question (V1) à two meaningful (0 &1) attributes and
one ‘missing’ category (999= no answer) à we want to store the answers
people give in a data matrix
If V1 = 0 or 999 à interview stops (V1 is a filter question) à in the next
questions they automatically get the number 998 (not asked)
V2 à has answers with a range between 0…200, 998 and 999 (a variable
with many different numerical values)
V3 à (if V2 is 999 the interview may still continue) Has answers
1,2,3,4,5,6,7,998 & 999 à because there can be several answers the
variable is stored as several variables à V3a, V3b, V3c etc.
V4 à open question à make it numerical
3.6 Missing values
Different types of missing values in surveys:
• Non-response à Did not participate in survey (normally non-response is not included in the data matrix)
• Filter Question (INAP) à Question not asked à Consequences: no loss of information
• Mistake by interviewer à Question not asked à Consequences: often loss of information
• Item non-response (NA) à Refuses to answer a question à Consequences: often loss of information
• Item non-response (DK) à Does not know the answer to a question à Consequences: sometimes loss
of information
Missing values may have very important consequences for the validity of the research à how do you correctly
handle missing values? à in SPSS tell the program it is a missing value à not all values in a data matrix should
be treated as ‘values.
9
, 3.7 The Practice of Social Research - Earl Babbie (Ch4 p. 97-105 + Ch5 p. 139-143)
A necessary cause is a condition that must be present to produce the effect: being female is a necessary cause
of being pregnant. A sufficient cause is one, which, when present, always causes the effect: skipping all the
exams in a course would be a sufficient cause of failing. Units of analysis are the people or things whose
characteristics social researchers observe, describe, and explain. Typically, the unit of analysis in social research
is the individual person, but it may also be a social group, a formal organization, a social interaction, a social
artifact, or some other phenomenon such as a lifestyle. The ecological fallacy involves taking conclusions drawn
solely from the analysis of groups (e.g., corporations) and applying them to individuals (e.g., the employees of
corporations). Reductionism is the attempt to understand a complex phenomenon in terms of a narrow set of
concepts, such as attempting to explain the American Revolution solely in terms of economics (or political
idealism or psychology).
Operationalization is an extension of conceptualization that specifies the exact procedures that will be used to
measure the attributes of variables. Operationalization involves a series of interrelated choices: specifying the
range of variation that is appropriate for the purposes of a study, determining how precisely to measure
variables, accounting for relevant dimensions of variables, clearly defining the attributes of variables and their
relationships, and deciding on an appropriate level of measurement. Researchers must choose from four levels
of measurement, which capture increasing amounts of information: nominal, ordinal, interval, and ratio. The most
appropriate level depends on the purpose of the measurement. A given variable can sometimes be measured at
different levels. When in doubt, researchers should use the highest level of measurement appropriate to that
variable so they can capture the greatest amount of information.
10
