Lecture 1: Fundaments of quantitative research; quantitative research designs
Social science methods
Methodological literacy
Understanding the principles of research methods
Learn to read critically
Practice using research methods
Why is social science so important?
Social science has made its way in any type of institution, domains, etc.
o Social science is used everywhere in society, for instance: :
Government policy
Education
Marketing:
The basis is always the same: Regression analysis, but it is done at a more advanced level
together with big data.
Political campaigns
Professional sports
Consultancy
Etc.
Why important for political science?
Especially in political science it is essential to evaluate scientific research and to understand its merits and limitations
Everyone has an opinion about politics: it is the scientific perspective what separates us/you from the mass….
o But, different perspectives on what social science should be (what can be reasonably known and observed)
This has produced two ‘branches’ in social sciences: a positivist/realist school relying mostly on quantitative data methods
(but not exclusively) and an interpretivist school relying on qualitative data methods (almost exclusively):
o Always can see two types of research: positivist embedded in advanced statistics and constructivism in line with
the interpretivist school.
Always keep in mind that these differences are vastly overblown, most researchers fall somewhere in the middle.
Ontology
Ontology
The goal of science is to learn about ‘reality’, but what is reality? Is there one reality or multiple (e.g. as many as there are
human beings?)
Ontology is the branch of philosophy (metaphysics) that focusses on these questions
The key ontological question in the social sciences is whether there is an objective reality outside the perspective of people
Two positions: constructivism and objectivism
Constructivism
Constructivism states that social reality is not the same as physical reality. People create their own reality through social
interactions.
A molecule might exists in reality, outside people’s mind, but something more abstract, such as culture, only gets meaning
in a person’s mind.
Social reality therefore can only be understood as the collection of perspectives in which the perspective of the researcher
is also ‘a perspective’. (‘the problem of the other mind’)
One perspective is not necessarily more valuable than others. The best we can do is ‘describe as thoroughly as possible the
perspectives of individuals’ and the social interactions that binds or divides them’ (=thick description)
,Objectivism
Objectivism is the ontological position that social observations are ‘real’: they exist outside a person’s mind..
Things such as ‘culture’ or ‘power’ are not only constructs of the mind, but ‘exists’ in the real world.
Objectivism is the ontological foundation for a positivist or realist epistemology.
Note: not a strict divide, but rather a scale...
Epistemology
Epistemology
Epistemology is ‘s the branch of philosophy concerned with the theory of knowledge, i.e. nature of knowledge, justification,
and the rationality of belief. ’
Resolves around the classical question: ‘what can I (learn to) know’
Epistemology is rooted in ontology (what you can learn depends on your view of reality)
A pertinent epistemological question for the social sciences is whether the methods used in physics are equipped to study
people, social interaction, and societies.
We are not going to answer this question here, but it is crucial that you understand that such questions are critical for the
type of more specific methods a researcher (like you) uses.
Three positions: interpretivism, positivism, realism
Interpretivism
Interpretivism states that there are vast differences between the methods social sciences and natural sciences rely on.
Interpretivism states that social science should focus on ‘verstehen’ instead of ‘erklären’ (Wilhelm Dilthey)
Erklären: Systematically study the conditions for certain events or relations.
Verstehen: Trying to understand why events happen or why social relations exist
According to interpretivists ‘verstehen’ is better suited for social science because the perspective of people is critical in
understanding their behavior.
Positivism
Positivism is the branch in social science which apply elements of the methodology from natural sciences to explain social
phenomenon (but adapted to cope with the social world): find trends in observable data.
According to positivists social sciences should focus on predicting observable phenomenon
o Focus and talk only about one thing that is observable and touchable. One such example is money, which can be
clearly observed.
Realism
Realism , in line with positivism, accepts the usefulness of the natural sciences methodology and accepts that social reality
is real and can be systematically studied.
Realism, however, accepts more room for ‘unobservable things’:
o Things that we cannot see, we cannot really measure, but still put attention on.
Realism is much more influential than strict positivism. For example, in political science we rely on many abstract concepts
which cannot be directly observed
Note: again, not a strict divide, but rather a scale.
,Theory and Empirics
Theory and Empirics
In scientific research you deal with theory and empirics
Theories are ideas about how things (reality) work. For instance:
o E = mc2
Einstein’s theory of special relativity.
o Longer prison sentences lead to less crime
o Fear for globalization leads people to vote for Donald Trump
Empirics is what we can factually observe in research. For instance:
o The speed of light which we observe
o Crime rates across countries
o Opinion polls on voting behavior
o Remember to always separate these in your research (i.e. assignments)!
Induction and Deduction
(Almost) every research includes theory and empirics, but where do you start?
Induction means you first observe reality, after which you try to order the results and based on this you describe a pattern
(formulate a theory): mostly used in interpretivism
Deduction means you first think about patterns in reality (‘theorize’) after which you check (do research) whether the
theory makes sense in reality (in the empirical world). Mostly used in positivism and realism
The principle of quantitative and qualitative research
The ‘defining feature’ of quantitative research is that reality is understood and described with the help of numbers (in
statistics) or words (in case studies); in qualitative research reality is interpreted through the perspective of the researcher
and people involved.
Quantitative research is therefore rooted in positivism or realism; yet qualitative research is (more often) rooted in
Interpretivism
Note: while we do see qualitative research in a positivist/realist tradition, qualitative research in a constructivist tradition is
really rare.
Distinction summary
,Quantitative research designs
Criteria for quantitative research (!! To be known by heart !!)
With this any type of argument can be broken down and countered. Three criteria and three types of research designs.
1. Reliability:
If you would replicate research would this lead to a similar outcome?
If research is not reliable, the findings could be random
2. Internal validity:
Is the causal inference claimed in the research valid?
o Can you claim that the causal story yu are telling is really accurate? Is there really a causal link?
If research is not internally valid, one cannot make a causal claim
3. External validity:
Do the results hold in a different context?
If results are not externally valid, the results do not say much about the ‘real world’
Is this a good case, a good sample, etc.
Quantitative research (and the first part of this course) is (are) founded on these three pillars!
When you choose a research design, evaluate each criteria.
Design
There are three designs to approach quantitative research: cross-sectional, longitudinal, and experimental
The first question you therefore should ask yourself when reading research is what is the design
Also, case studies and comparative research have only one of these type of designs
Generally speaking, the controlled experiment is the foundation for scientific research. And some political scientists use experiments
in their work. However, owing to the nature of our subject matter, most political scientists adopt one of two types of "observational"
research designs that are intended to mimic experiments.
Comparison as the key to establishing causal relationships
Most phenomena we are interested in explaining have multiple causes, but our theories typically deal with only one of them while
ignoring the others. The multivariate nature of the world can make our first glances at evidence misleading.
Comparisons are at the heart of science. If we are evaluating a theory about the relationship between some X and some Y, the
scientist's job is to do everything possible to make sure that no other influences (Z) interfere with the comparisons that we will rely
on to make our inferences about a possible causal relationship between X and Y.
To test whether X really causes Y, there are several strategies or research designs that researchers can use toward that end. The goal
of all types of research designs is to help us evaluate how well a theory fares as it makes its way over the four causal hurdles - that is,
to answer as conclusively as is possible the question about whether X causes Y.
Observational studies are not experiments, but they seek to emulate them. They are known as observational studies
because, unlike the controlled and somewhat artificial nature of most experiments, in these research designs, researchers
simply take reality as it is and "observe" it, attempting to sort out causal connections without the benefit of randomly
assigning participants to treatment groups. Instead, different values of the independent variable already exist in the world,
and what scientists do is observe them and then evaluate their theoretical claims by putting them through the same four
causal hurdles to discover whether X causes Y.
The definition of an observational study: An observational study is a research design in which the researcher does not have
control over values of the independent variable, which occur naturally. However, it is necessary that there be some degree
of variability in the independent variable across cases, as well as variation in the dependent variable.
, Within observational studies, there are two pure types
1. Cross-sectional research and designs
In a cross-sectional design one compares variables in one moment in time:
o Examines a cross section of social reality, focusing on variation between individual spatial units - again, like citizens,
elected officials, voting districts, or countries - and explaining the variation in the dependent variable across them.
In a typical cross-sectional design (for instance with voters as the unit of analysis) a large group of people is surveyed with
closed (multiple choice) questions or social reality is coded.
o E.g.: How positive is your attitude towards migrants on a scale of 1 to 10?
o E.g.: What is your length?
o E.g. How democratic is a country
o E.g. How many war victims are there in a country
Every questions in the survey is a variable: something on which respondents can differ
o E.g. Attitude towards migrants, height, level of democracy, war victims
Cross-sectional research is often called correlation-research because one looks for correlations between variables.
o E.g.: If men are taller than women, this means there is a correlation between gender and height of people
o E.g. if there are less war victims in democratic countries this means there is a correlation between democracy and
war victims
Advantages and disadvantages cross-sectional designs
High reliability:
o In a cross-sectional design many people can participate (i.e. a large sample can be drawn) which increases the
chance that findings are random.
o If you, for instance, find that men are taller than women in a survey of 5000 participants (randomly selected), the
chances is very high that if you would replicate this study you would come to similar results (even if it is a different
sample).
High external validity:
o It is relatively easy to find a representative sample in a cross-sectional design.
o The participants of the study should share many characteristics with the population, for instance their age, gender,
occupation, opinions, etc.
o This means that for this type of research, the chances are high that what you find can be generalized to the
population:
If the sampling is random.
Downside: lower internal validity
o It is harder to make causal claims because the results can be spurious (a statistical correlation without a causal link)
or there is endogeneity (i.e. reverse causality).
But, there are ways to increase the internal validity
o For spurious relation: including control variables
o For endogeneity: strong theory or addition of other research designs (i.e. longitudinal or experiment)
2. Longitudinal research and designs
Longitudinal research
Longitudinal research is similar to cross sectional research with one key difference: the same people are surveyed on
different moments in time.
o If you apply a statistical analysis with this type of data you can much clearly see where the causal relation lies.
, A well-known example is a panel of people which participate in a survey each year (e.g. follow lobbyists over time)
In political science we often do longitudinal research in which countries are compared over time (for instance to trace the
development of democracies)
Benefits longitudinal research
Better internal validity than cross-sectional design:
o By analyzing the same people (or countries) over time you come one step closer to cause-effect relations because
you have a sequence in time and you can control for associations in groups.
Sequence in time:
If you follow someone (or somewhat) over time you can see wat follows what: something can
only cause something if it happens before the other… (see endogeneity)
Correlations within groups:
If you follow someone (or somewhat) over time you can see whether something else happens
during the process (see spurious relation)
Sometimes more data than cross sectional research:
This is especially relevant for countries, which is simply a limited source of data (195 countries in the world)
Downsides longitudinal research
Internal validity better than cross sectional
o Longitudinal design offers more chances to establish a cause-effect relationship than cross-sectional design, but
variables do not always vary consistently (can still be problematic to exclude endogeneity) and you still need to
control for third variables (to exclude spurious relationship)
Reliability tends to be somewhat lower than in a cross-sectional design
o It is often harder to find participants to cooperate in a longitudinal design, while the statistical models are more
complex. This increases the chances that replication on a different sample would produce (somewhat) different
results.
o It is hard to find the same people and convince them to answer to the same questions year after year.
Lower external validity than cross-sectional design
o There are always participants that drop out (this is called attrition). Because some type of groups tend to drop out
more frequently (lower educated people) it is more difficult to generalize to the entire population.
o For countries this is of course different (here data availability is often a problem)
3. Experimental designs
Idea
An experimental design is specifically designed to capture causal mechanisms:
o Kellstedt and Whitten: An experiment is a research design in which the researcher both controls and randomly
assigns values of the independent variable to the participants.
The ‘defining features’ of an experiment are manipulation and randomization
o Manipulation: the researchers changes something, for instance a particular type of medicine, in one group
(treatment group) and not in another group (control group)
Randomly select a group of people and then split them into two randomized groups. One group gets the
treatment, the other does not.
What does it mean to say that a researcher "controls" the value of the independent variable that the
participants receive? It means, most importantly, that the values of the independent variable that the
participants receive are not determined either by the participants themselves or by nature.
o Randomization: who gets the stimulus (e.g. the medicine) and who doesn’t is randomly decided by chance:
Why randomization?
, o To decide whether a stimulus works (e.g. medicine) you want to compare groups which differ in only one way: the
stimulus.
o In all other ways the groups should be similar (age, gender, education, etc). This is the only way that you know the
stimulus has an effect.
o More abstract: Only through randomization every participant of the experiment has an equal chance to be in the
treatment group or the control group.
What the experiment does, through the process of randomly assigning subjects to different values of X, is
to equate the treatment and control groups on all possible factors. On every possible variable, whether or
not it is related to X, or to Y, or to both, or to neither, the treatment and control groups should, in theory,
be identical. That makes the comparison between the two values of X unpolluted by any possible Z
variables because we expect the groups to be equivalent on all values of Z.
o Random sampling:
Random sampling: how researchers select cases for inclusion in a study – they are selected at random, so
every member of the underlying population has an equal probability of being selected.
First, we should evaluate whether there is a credible causal mechanism before we decide to run the experiment.
Benefits experimental research
High internal validity
o Reversed causality can be excluded (endogeneity) through manipulation
o Spurious relation can be excluded by randomization
o Through manipulation and randomization, you can better grasp a cause-effect relationship
Experiments is the gold standard in natural sciences. Medicines are only approved if they have been tested in experiments.
Weaknesses experimental design
Lower reliability:
o For practical reasons normally very low number of participants:
It is hard to find people and convince them to come to the laboratory and so on.
o Chances are high if you replicate the study you come to different outcomes
Not every independent variable (X) is controllable and subject to experimental manipulation
Lower external validity:
o It is very hard to find a representative sample of the population (e.g. extreme bias to students)
The participant pool in this case represents what we would call a sample of convenience, which is to say,
this is more or less the group of people we could beg, coerce, entice, or cajole to participate.
o The experiments are done in an artificial environment which differs from their natural habitat (External validity of
the stimulus)
Ethical issues:
o They carry special ethical dilemmas for the researcher. Ethical issues about the treatment of human participants
occur frequently with medical experiments.
