General topics
Which types of research can be conducted with observations, correlations, and
experiments, respectively?
o Observations —> finding phenomena
o correlations —> finding relationships
o experiments —> finding causal relationships
o all of them —> testing theories of experience and behavior
What is meant by the precision of a theory?
o That an experiment should make precise predictions
What is meant by the parsimony of a theory?
o That you choose the simplest theory that fits with the evidence
Why are testability and falsifiability considered important features of a theory?
o It must be possible to test and prove a theory wrong. if a theory is always right, it
is worthless.
What is the internal validity of a study?
o If the intervention rather than a confounded variable caused the results
What is the external validity of a study?
o How far can the results be generalized?
What is the construct validity of a study?
o Which aspects of the intervention caused the results?
What is the statistical validity of a study?
o Are the statistical conclusions, correct? most often the statistics are right, but
the conclusions are wrong.
How can correlations be used and interpreted?
o Correlation: direction and size, regression: prediction
How can correlations not be interpreted?
o Just because one variable predicts the other, does not mean that it causes the
other.
Does correlation imply causality? If yes, why? If not, why not?
o No, it does not. the relation is not symmetric.
Does causality imply correlation? If yes, why? If not, why not?
o Yes, causality implies correlation
How does the temporal order of two variables help to establish a causal relation
between them?
o Temporal order does not prove causality. If A is the cause of B, A must happen
before B.
o Untrue: If A happens before B, then A is the cause of B. THIS IS UNTRUE. it
can be right but does not have to be.
o Correlation is a necessary, but not a sufficient precondition for causation!!
What do you have to do to test whether two variables are causally related?
o Conduct an experiment!
What are independent, dependent, and control variables of experiments?
o Independent variable is manipulated by the experimenter
How many levels are there of the variable?
, o Dependent variable is measured by the experimenter (behavior, subjective
experience, physical response)
Beware of the floor effects and ceiling effects! (everyone behaves the
same way)
o Control variables are controlled by the experimenter
hold them constant
turning them into independent variables
What does it mean if an experimental independent variable is a between-subjects
variable?
o every subject experience only one level of the independent variable random
assignment!
o by using random assignment, you prevent all the pitfalls of an experiment
o random assignment only works if you have two large groups (1000+) then
there will be no significant differences between the two groups. with every group
that is smaller, you have to check the small things (age)
What does it mean if an experimental independent variable is a within-subjects
variable?
o every subject experience every level of the independent variable: order effects?
o no groups have to be equal because everyone experiences everything
o maybe the order is important, so use random assignment to assign half the group
to one variable first, and the other group to experience the other variable first
then, compare the outcomes.
What are advantages and disadvantages of between-subjects and within-subjects
experimental designs?
o disadvantages:
we have to take people as they come this is no longer an experiment; it
is quasi-experimental.
you can say that there is a relationship, but you cannot determine
the cause of this relationship
if we do lab research, we don’t know if it is generalizable to everyday life
(external validity)
we cannot generalize from an analogue population to a general population
low sample size low statistical power
What is random assignment, and why is it so very important?
o when you randomly assign a population to a level/variable. it is really important
because you have no influence on who experienced what. it makes it more
generalizable to a population, only if you are using large sample sizes (1000+)
o it reduces selection and allocation biases. all groups are initially similar on
observed and unobserved characteristics
What is the difference between a quasi-experiment and a real experiment?
o quasi experiments do not rely on random assignment. subjects are assigned to
groups based on non-random criteria
Statistical Power
In statistical testing, what is the alpha error?
, o False positive conclusion rejecting null hypothesis when it is actually true
In statistical testing, what is the beta error?
o false negative conclusion failing to reject null hypothesis when it is false
What does the term "effect size" mean?
o a number that illustrates how big an effect is (difference, correlation, relationship)
What does the term "statistical power" mean?
o what is the probability that this effect will be statistically significant in an
experiment?
o different experiments will have different power
What does it mean when the statistical power of a study is small or large,
respectively?
o
In which situations is it important to consider the power of studies?
o
How is Cohen's effect size value d computed?
o difference between the two means of the two groups of the two conditions,
divided by the standard deviation.
What are the conventional values of d for small, medium, and large effects,
respectively?
o small: 0,2
o medium: 0,5 (because we usually find it
o large: >0,8 (doesn’t really happen)
Which factors affect the statistical power of a study?
o effect size:
larger effects are easier to find
o sample size
effects are easier to find with many participants
o alpha error
increasing the alpha error (type I) reduces beta error (type II)
usually not an alternative
How can the power of a study be increased?
o effect size:
larger effects are easier to find
o sample size
effects are easier to find with many participants
Which effect size values are usually used together with t-tests, ANOVAs, and
correlations?
o t-Test: d
o ANOVA:
f (f=d/2)
partial eta2 (percentage of explained variance)
o correlation: r (Pearson’s correlation coefficient)
Why is the correlation a particularly simple and useful effect size value?
o the nearer it gets to 1 or -1, the larger your effect size the stronger the relation
is between the two variables that you are correlating
Which types of research can be conducted with observations, correlations, and
experiments, respectively?
o Observations —> finding phenomena
o correlations —> finding relationships
o experiments —> finding causal relationships
o all of them —> testing theories of experience and behavior
What is meant by the precision of a theory?
o That an experiment should make precise predictions
What is meant by the parsimony of a theory?
o That you choose the simplest theory that fits with the evidence
Why are testability and falsifiability considered important features of a theory?
o It must be possible to test and prove a theory wrong. if a theory is always right, it
is worthless.
What is the internal validity of a study?
o If the intervention rather than a confounded variable caused the results
What is the external validity of a study?
o How far can the results be generalized?
What is the construct validity of a study?
o Which aspects of the intervention caused the results?
What is the statistical validity of a study?
o Are the statistical conclusions, correct? most often the statistics are right, but
the conclusions are wrong.
How can correlations be used and interpreted?
o Correlation: direction and size, regression: prediction
How can correlations not be interpreted?
o Just because one variable predicts the other, does not mean that it causes the
other.
Does correlation imply causality? If yes, why? If not, why not?
o No, it does not. the relation is not symmetric.
Does causality imply correlation? If yes, why? If not, why not?
o Yes, causality implies correlation
How does the temporal order of two variables help to establish a causal relation
between them?
o Temporal order does not prove causality. If A is the cause of B, A must happen
before B.
o Untrue: If A happens before B, then A is the cause of B. THIS IS UNTRUE. it
can be right but does not have to be.
o Correlation is a necessary, but not a sufficient precondition for causation!!
What do you have to do to test whether two variables are causally related?
o Conduct an experiment!
What are independent, dependent, and control variables of experiments?
o Independent variable is manipulated by the experimenter
How many levels are there of the variable?
, o Dependent variable is measured by the experimenter (behavior, subjective
experience, physical response)
Beware of the floor effects and ceiling effects! (everyone behaves the
same way)
o Control variables are controlled by the experimenter
hold them constant
turning them into independent variables
What does it mean if an experimental independent variable is a between-subjects
variable?
o every subject experience only one level of the independent variable random
assignment!
o by using random assignment, you prevent all the pitfalls of an experiment
o random assignment only works if you have two large groups (1000+) then
there will be no significant differences between the two groups. with every group
that is smaller, you have to check the small things (age)
What does it mean if an experimental independent variable is a within-subjects
variable?
o every subject experience every level of the independent variable: order effects?
o no groups have to be equal because everyone experiences everything
o maybe the order is important, so use random assignment to assign half the group
to one variable first, and the other group to experience the other variable first
then, compare the outcomes.
What are advantages and disadvantages of between-subjects and within-subjects
experimental designs?
o disadvantages:
we have to take people as they come this is no longer an experiment; it
is quasi-experimental.
you can say that there is a relationship, but you cannot determine
the cause of this relationship
if we do lab research, we don’t know if it is generalizable to everyday life
(external validity)
we cannot generalize from an analogue population to a general population
low sample size low statistical power
What is random assignment, and why is it so very important?
o when you randomly assign a population to a level/variable. it is really important
because you have no influence on who experienced what. it makes it more
generalizable to a population, only if you are using large sample sizes (1000+)
o it reduces selection and allocation biases. all groups are initially similar on
observed and unobserved characteristics
What is the difference between a quasi-experiment and a real experiment?
o quasi experiments do not rely on random assignment. subjects are assigned to
groups based on non-random criteria
Statistical Power
In statistical testing, what is the alpha error?
, o False positive conclusion rejecting null hypothesis when it is actually true
In statistical testing, what is the beta error?
o false negative conclusion failing to reject null hypothesis when it is false
What does the term "effect size" mean?
o a number that illustrates how big an effect is (difference, correlation, relationship)
What does the term "statistical power" mean?
o what is the probability that this effect will be statistically significant in an
experiment?
o different experiments will have different power
What does it mean when the statistical power of a study is small or large,
respectively?
o
In which situations is it important to consider the power of studies?
o
How is Cohen's effect size value d computed?
o difference between the two means of the two groups of the two conditions,
divided by the standard deviation.
What are the conventional values of d for small, medium, and large effects,
respectively?
o small: 0,2
o medium: 0,5 (because we usually find it
o large: >0,8 (doesn’t really happen)
Which factors affect the statistical power of a study?
o effect size:
larger effects are easier to find
o sample size
effects are easier to find with many participants
o alpha error
increasing the alpha error (type I) reduces beta error (type II)
usually not an alternative
How can the power of a study be increased?
o effect size:
larger effects are easier to find
o sample size
effects are easier to find with many participants
Which effect size values are usually used together with t-tests, ANOVAs, and
correlations?
o t-Test: d
o ANOVA:
f (f=d/2)
partial eta2 (percentage of explained variance)
o correlation: r (Pearson’s correlation coefficient)
Why is the correlation a particularly simple and useful effect size value?
o the nearer it gets to 1 or -1, the larger your effect size the stronger the relation
is between the two variables that you are correlating
