Research Methods
Week 1 Readings:
The Fundamentals of Political Science Research (Kellstedt and Whitten) CH4:
➔ Generally speaking, the controlled experiment is the foundation for scientific
research. And some political scientists use experiments in their work. However,
owing to the nature of our subject matter, most political scientists adopt one of two
types "observational" research designs that are intended to mimic experiments. The
cross-sectional observational study focuses on variation across individual units (like
people or countries). The time-series observational study focuses on variation in
aggregate quantities (like presidential popularity) over time.
➔ An experiment is a research design in which the researcher both controls and
randomly assigns values of the independent variable to the participants.
➔ The researcher both controls values of the independent variable-or X, as we
have called it - as well as randomly assigns those values to the participants in
the experiment.
➔
➔ What experimental research designs accomplish by way of random assignment to
treatment groups, then, is to decontaminate the comparison between the treatment
and control group of all other influences. Before any stimulus (like a treatment or
placebo) is administered, all of the participants are in the same pool. Researchers
divide them by using some random factor like a coin flip, and that difference is the
only difference between the two groups.
➔ Coming up with a credible causal scenario that links X to Y heightens our
dependence on theory, not on data or research design.
➔ It is critical that you do not confuse the experimental process of randomly assigning
subjects to treatment groups, on the one hand, with the process of randomly
sampling subjects for participation, on the other hand.
➔ Random assignment to treatment and control groups occurs when the
participants for an experiment are assigned randomly to one of several possible
values of X, the independent variable. Importantly, this definition says nothing at all
about how the subjects were selected for participation.
➔ random sampling is, at its very heart, about how researchers select cases for
inclusion in a study - they are selected at random, which means that every member
of the underlying population has an equal probability of being selected. (This is
common in survey research, for example.).
,➔ To use computers in the interviewing process that includes experimental
randomization of variations in survey questions, in a technique called a survey
experiment. Such designs are intended to reap the benefits of both random
assignment to treatment groups, and hence have high internal validity, as well
as the benefits of a random sample, and hence have high external validity.
➔ A field experiment is one that occurs in the natural setting where the subjects
normally lead their lives. Random assignment to treatment groups has enabled
researchers in the social sciences to study subjects that seemed beyond the reach of
experimentation.
➔ specially in the social sciences, not every independent variable (X) is controllable
and subject to experimental manipulation. There are simply a myriad of substantive
problems that are impossible to study in an experimental fashion. Social scientists
cannot, in any meaningful sense, "assign" people.
➔ Experiments often suffer from low degrees of external validity. Sample of
convenience, which is to say, this is more or less the group of people we could beg,
coerce, entice, or cajole to participate. With a sample of convenience, it is simply
unclear how, if at all, the results of the experiment generalize to a broader
population. In our hypothetical experiment, the individual does not choose what he
or she sees. The exposure to the ad is forced (once the subject consents to participate
in the experiment). At home? People who don't want to be exposed to political ads
can avoid them rather easily if they so choose, simply by not watching particular
channels or programs, or by not watching TV at all, or by flipping the channel when
a political ad starts up. But the comparison in our hypothetical experiment is
entirely insensitive to this key difference between the experimental environment
and the subject's more natural environment.
➔ they carry special ethical dilemmas for the researcher.
➔ When interpreting the results of an experiment, we sometimes make mistakes of
emphasis. If an experiment produces a finding that some X does indeed cause Y, that
does not mean that that particular X is the most prominent cause of Y.
➔ Observational studies are not experiments, but they seek to emulate them. They are
known as observational studies because, unlike the controlled and somewhat
artificial nature of most experiments, in these research designs, researchers simply
take reality as it is and "observe" it, attempting to sort out causal connections
without the benefit of randomly assigning participants to treatment groups. Instead,
different values of the independent variable already exist in the world, and what
scientists do is observe them and then evaluate their theoretical claims by putting
them through the same four causal hurdles to discover whether X causes Y.
➔ An observational study is a research design in which the researcher does not have
control over values of the independent variable, which occur naturally. However, it
is necessary that there be some degree of variability in the independent variable
across cases, as well as variation in the dependent variable.
➔ observational studies have to cross the same four casual hurdles as do experiments.
And we have noted chat, unlike experiments, with their random assignment to
treatment groups, observational studies will often gee stuck on our fourth hurdle.
➔ Cross-sectional and time-series approaches are both useful in this respect. They
simply address different types of substantive questions. Cross-sectional approaches
look to see why some individuals prefer more liberal government policies, and why
some other individuals prefer more conservative government policies.
,Lecture 1:
ONTOLOGY, EPISTEMOLOGY, METHODOLOGY
Ontology:
➔ The goal of science is to learn about ‘reality’, but what is reality ? Is there one reality
or multiple realities (e.g. as many as there are human beings?)
➔ Ontology is the branch of philosophy (metaphysics) that focuses on these questions
➔ The key ontological question in the social sciences is whether there is an objective
reality outside the perspective of people
Ontology: Constructivism
➔ Constructivism states that social reality is not the same as physical reality. People
create their own reality through social interactions
➔ A molecule might exist in reality, outside people’s minds, but something more
abstract, such as culture, only gets meaning in a person’s mind
➔ Social reality therefore can only be understood as the collection of perspectives in
which the perspective of the researcher is also ‘a perspective’ (the problem of the
other mind)
➔ One perspective is not necessarily more valuable than another. The best we can do
is ‘describe as thoroughly as possible the perspectives of individuals and the social
interactions that bind or divide them’ (think description)
Ontology: Objectivism
➔ Objectivism is the ontological position that social observations ae ‘real’: they exist
outside a person’s mind
➔ Things such as ‘culture’ or ‘power’ are not only constructs of the mind, but ‘exist’ in
the real world
➔ Objectivism is the ontological foundation for a positivist or realist epistemology
➔ Note: not a strict divide but rather a scale
Epistemology:
➔ Epistemology is the branch of philosophy concerned with the theory of knowledge,
i.e. nature of knowledge, justification, and the rationality of belief
➔ Revolves around the classical question: ‘what can I (learn to) know
➔ A pertinent epistemological question for the social sciences is whether the methods
used in physics are equipped to study people, social interaction, and societies
➔ We are not going to answer this question here, but it is crucial that you understand
that such questions are critical for the tat such questions are critical for the type of
more specific methods a researcher (like you) uses.
➔ Or at least, it (partly) explains why this method course is divided in two parts
Epistemology: Interpretivism
➔ Interpretivism states that there are vast differences between the methods social
sciences and natural sciences rely on
➔ Interpretivism states that social science should focus on ‘verstehen’ instead of
‘erklaren” (Wilhelm Dilthey)
➔ Erklaren: Systematically study the conditions for certain events or relations
➔ Verstehen: Trying to understand why events happen or why social relations exist
, ➔ According to interpretivists, verstehen is better suited for social science because the
perspective of people is critical in understanding their behaviour
Epistemology: Positivism
➔ Positivism is the branch in social science which apply elements of the methodology
from natural sciences to explain social phenomena (but adapted to cope with the
social world): find trends in observable data
➔ According to positivists social sciences should focus on predicting observable
phenomena
Epistemology: Realism
➔ Realism, in line with positivism, accepts the usefulness of the natural sciences
methodology and accepts that social reality is real and can be systematically studied
➔ Realism, however, leaves more room for ‘unobservable things’
➔ Realism is much more influential than strict positivism. For example, in political
science we rely on many abstract concepts which cannot be directly observed
➔ Note: Not a strict divide, but rather a scale
THEORY AND EMPIRICS
➔ In scientific research you deal with theory and empirics
➔ Theories are ideas about how things work. (example: Longer prison sentences lead
to less crime)
➔ Empirics is what we can actually observe in research. (example: Crime rates across
countries)
➔ Remember to always separate these in your research
INDUCTION AND DEDUCTION
➔ (Almost) every research includes theory and empirics, but where do you start ?
➔ Induction means you first observe reality, after which you try to order the results
and based on this you describe a pattern (formulate a theory). Mostly used in
interpretivism
➔ Deduction means you first think about patterns in reality (theorize) after which you
check (do research) whether the theory makes sense in reality (the empirical
world). Mostly used in positivism and realism
QUANTITATIVE AND QUALITATIVE RESEARCH
➔ The defining feature of quantitative research is that reality is understood and
described with the help of numbers (in statistics) or words (in case studies); in
qualitative research reality is interpreted through the perspective of the researcher
and people involved
➔ Quantitative research is therefore rooted in positivism and realism; qualitative
research is (more often) rooted in interpretivism
➔ Note: while we do see qualitative research in a positivist/ realist tradition,
qualitative research in a constructivist tradition is really rare
