Exam 2 information:
Morling
Chapter 3, chapter 6, chapter 7 and chapter 10
Agresti
Chapter 6, chapter 7 and chapter 8 (except paragraph 8.5)
Chapter 3 - Three Claims, Four Validities: Interrogation Tools for Consumers of
Research (Morling)
Variables
A variable is a thing that can take multiple “values”. More specifically, it’s described
in the book that a variable is any characteristic observed in a study, as something that varies.
A measured variable (or dependent variable) is a variable whose levels are simply observed
and recorded. A manipulated variable (or independent variable) is a variable a researcher
controls. Conceptual variables are abstract concepts, this is sometimes called a construct.
Conceptual variables must be carefully defined at the theoretical level, and these definitions
are called conceptual definitions. They turn these concepts of interest into a measured or
manipulated variable, known as operationalizing. They create operational definitions of the
variables, also known as operational variables.
Three Claims
A claim is an argument someone is trying to make. Frequency claims describe a
particular rate or degree of a single variable (e.g. “2 out of 5”, “25% says…”). Frequency
claims focus on one variable. In studies that support frequency claims, the variables are
always measured, not manipulated. An association claim argues that one level of a variable
is likely to be associated with a particular level of another variable. They are said to correlate.
An association claim states a relationship between at least two variables. In a study that
measures variables and makes a claim about whether or not two variables correlate, is called
a correlation study. A causal claim argues that one of the variables is responsible for
changing the other variable. Causal claims are a step above association claims. To move from
association to causality, a study has to satisfy three criteria.
1) It must establish that the two variables are correlated, the relationship
cannot be zero. -> Covariance / Statistical validity
2) It must show that the causal variable came first an the outcome variable
came later. -> Temporal precedence.
3) It must establish that no other explanation exists for the relationship.
Only an experiment will support a causal claim. -> Internal validity.
,Interrogating the Three Claims Using the Four Big Validities
Validity refers to the appropriateness of a conclusion or decision, and in general, is a
valid claim is reasonable, accurate, and justifiable. The four big validities are:
1) Construct validity - How well is the conceptual variable operationalized?
2) Statistical validity - The extent to which a study’s statistical conclusions are
accurate and reasonable.
3) Internal validity - refers to how well an experiment is done, especially
whether it avoids confounding (more than one possible independent variable [cause] acting at
the same time).
4) External validity - How well do the results of a study generalize to, or
represent, people or contexts beside those in the original study?
Interrogating Frequency Claims
When evaluating the construct validity of a frequency claim, the question is how well
the researchers measured their variables. When asking how well a study measured or
manipulated a variable, you are interrogating the construct validity. To ensure construct
validity, the researchers must prove the variables to be measured reliably, so in other words
that several testings showed similar results. Generalizability is whether or not the results of
the study are transferable from the specific group used to the general population. This is
important for the external validity. For the statistical validity, we ask ourselves how well the
numbers support the claim. The percentage reported in a frequency claim is usually
accompanied by a margin of error of the estimate, which should give you a solid idea.
Interrogating Association Claims
To support an association claim, a researcher measures two variable, so you have to
assess the construct validity of each variable. If you conclude one of the variables was
measures poorly, you would not be able to trust the conclusions related to that variable. You
might also interrogate the external validity of an association claim b asking whether it can
generalize to other populations, as well as to other contexts, times, or places. When applied to
an association claim, statistical validity is the extent to which the statistical conclusions are
, accurate and reasonable. One aspect of statistical validity is strength, so how strong the the
association is. Another question worth interrogating is the statistical significance of a
particular association, because some associations might be due to chance. Now comes the
hard part. There is a Type I error an a Type II error. In the book, a type I error is described
as a “false positive”. In the book they give the example of finding an association between two
variables in a sample, but no association in the population. It is also known as rejecting a true
null hypothesis. A null hypothesis is a hypothesis that states that there is no association or
relationship between two variables. Rejecting a null hypothesis is assuming there is a
relationship. So a Type I error is rejecting the null hypothesis (there is no relationship),
assuming there is an association, even though there is none. A Type II error is described in
the book is assuming there is no relationship or association between two variables, because it
didn’t show in the sample, but seeing an association in the population. This is known as a
“miss”. This is when we fail to reject a false null hypothesis, so there is no evidence in the
sample that there is an association, but we see one in the population.
Interrogating Causal Claims
We already discussed the three criteria for causation, but just because the books and
Dylan can’t seem to stress it enough, let’s have a reminder:
1) Covariance - The extent to which two variables are observed to go
together. One variable usually cannot be said to cause another variable unless the two are
related. The relationship cannot be zero.
2) Temporal precedence - One variable comes first in time, before the other
variable. It must show that the causal variable came first and the outcome variable came later.
3) Internal Validity, or third-variable criterion, - There is no other
explanation for the relationship. It eliminates alternative explanations for the association.
To support a causal claim, researchers must conduct an experiment. The manipulated
variable is the cause and the outcome is the measures variable. The manipulated variable is
called the independent variable, and the measures variable is called the dependent variable.
Random assignment is the, perhaps you guessed it, randomly assignment of participants into
groups, so the groups are similar. This is a way to control potential alternative explanations.
Types of Research and their claims
1) Surveys and/or interview research
2) Observational research
3) Correlational research
Studies association between two variables.
4) Experimental research
Manipulate the cause and study the effect.
