Lecture 1 Experimental Research
Phase 1: Formulaton problem statement and hypotheses about the relatonships, this
phase deals with the formulaton of the problem statement and hypotheses about the
relatonship between an iv and a dv. Wee will learn how to generate a researchable problem
statement together with the specifc hypotheses that will be tested in an experiment.
Phase 2: Design of the experiment, deals with the design of an experiment in which you 1.
manipulate an iv to observe efects on dv and 2. control for confounding factors. Wee will
discuss a. how the iv and dv are operatonaliied in experiments, and b. how to control for
cofounding variables.
Phase 3: Conductng the experiment and collectng data, this phase deals with conductng
the experiment and collectng data. Wee will discuss diferent types of experimental designs
and understand the diferences between main and interacton efects.
Phase 4: Data analysis and interpretaton, The fnal phase deals with the data analysis and
interpretaton of the experimental fndings. Wee will discuss a. how to statstcally analyie
experimental designs using ANOVA techniques, b. how to interpret results obtained from an
experiment, and c. how these results can lead to derive new hypotheses to be tested in a
follow-up experiment.
Why do we do research?
Goal: Describing behavior, predictng behavior and explaining behavior.
Describe behavior: large studies of randomly selected respondents to determine what
people think, feel, and do (e.g., opinion polls on politcss. arketng researchers conduct
descriptve research to study consumer preferences.
Predict behavior: Wee create diferent versions of a product, and we test which people like
best, in order to predict which will be most successful
Explaining behavior: many researchers see this as the most important goal of research.
any researchers feel that they do not really understand untl they can explain it.
Types: Descriptve, correlatonal, experimental and quasi experimental to really predict
and explain we use experimental research
Descriptve Research:
- Public opinion polls = most common example: 51% say this, 43% say this.
- Survey Research = respondents provide informaton on themselves by completng a
questonnaire or answering an interviewer’s questons
- Changes can be measured, if respondents fll out the survey at diferent points in
tme (longitudinal or panel design = more complex design. E.g. experiment about
greed: older people seem to be less greedy than younger, is this because they are
older or because they are from another tme cohort.
Correlatonal Research:
- Investgates the relatonships among diferent groups. Example experiment Howard
and Hofman, when the weather is beter, people are happier. This actually is not a
, good example because it could be that people do things that will make them happier
when the weather is beter (like drinking outside in the suns.
- Its aim is to discover correlatons between variables
Another example is the papier from Zwebner, Lee and Goldenberg: correlaton
between temperature and willingness to pay, are there other things that can
infuence the dv eesr
- Spurious correlatons: relatonships that do not exist. For example: when the weather
is beter people are more aggressive but also ice cream sales goes up.
Three requirements for correlatons
- Correlaton is only one of the necessary conditons for causality
- Directonality (logical in tmes
- Eliminaton of extraneous variables
Experimental research
Only if we are manipulatng an IV and assess the efect on DV, we can speak of an
experimentr
Randomizaton
- Arbitrarily assigning each partcipant to one conditon of the experiment (randomlys
- If I assign person A to the control group, and assign person B to a treatment group,
person A and B are not the same
- However, if I do this for every subject, the average person in the control group is the
same as the average person in the treatment group
- Weith large samples, true randomiiaton creates balance in for example age, gender,
preference of partcipants. The groups are thus the same on average
- Any diference we thus later fnd must have occurred because of the treatment.
As the groups are the same on average on all aspects it is beter than for example matching
techniques.
Why?
Because for matching we have to a priori predict the possible confounds (alternatve
explanatonss and match people based on those confounds. However, there could well be
confounds that we do not yet know about.
Quasi-Experimental research
In some cases it might not be possible to manipulate (changes the independent variable,
quasi-experimental research is used. E.g. test with fatal trafc accidents and seat belts.
For papers: we do not need to know the details etc. But we need to no:
- Wehy they did the study
- Wehat is the hypothesis. For e.g. experiment 1, paper 1: why did they did that
experiment: to see if the efect is real, is it out there in the real world.
A good explanaton of why something happens is a THEORe
,A good theory:
- Defnes constructs
o Paper example:
X = Physical warmth, is the independent variable
Conceptual defniton: Feeling warm (as compared to feeling
colds
Operatonal defniton: Daily temperature, Holding a warm vs.
cold object. Being in a warm vs. cold room
e = Product evaluaton, dependent variable
Conceptual defniton: subjectve evaluaton of a target product
Operatonal defniton: likelihood to purchase, willingness to
pay
- Propositons about the relatonships between constructs
o Paper example: physical warmth positve mood higher product
evaluatons
o Because: physical warmth creates a beter mood, and in a positve mood we
see other things in a more positve light
- Arguments that justfy the propositons
o E.g. paper: when experiencing warmth, people like other people more
o E.g. paper: people ofen treat objects (productss as if they were people
Statstcs
The base: What is a p-value?
Are diferences we see in the data likely due to error variance
A low p-value provides an indicaton that the H0 (the hypothesis that there is no efects is
unlikely
P=0.004 thus means that when there really is no efect, there is only a 0.4% chance that we
would have found the data that we did.
So technically, it does not mean that our hypothesis that X afects e is truer
Common mistakes with p-values
Do not round p-values, because you throw away valuable informaton
Only do this: for p’s < 0.001. eou basically say, ‘this is a very small p-value’.
If you have to present too many p-values in for example a correlaton table where you can
use a * to indicate p<0.05, ** for p<0.01 and *** for p< 0.001.
Wehat does marginally signifcant mean: e.g., p = 0.071: we are not really sure yetr It simply
means that the chance of the fnding being caused by error is more likely.
The base
If we observe a diference on a measure between two groups, what makes us more likely to
believe that there actually is a diference
, - A larger diference between the groups
- Less variaton between individuals in each group
- A larger sample siie
(KNOW FORMULA) Standard deviaton: is close to the average absolute deviaton
from the mean. (How much does it difer from the means. *But not quite because
of the square and square root (which basically increase the estmate due to
weighing relatie outliers more, compared to if we had only used absolute
diferences.
(KNOW FORMULA) T-test: test whether there is a diference between 2 means
In other words, all else being equal:
- A larger diference between means gives a larger t-value
- A smaller variance gives a larger t-value
- A larger sample siie gives a larger t-value
We have a t-value. Now what?
Recall that we are looking for a p-value as an indicaton of the likelihood that the null
hypothesis is false: how likely is it that we see this data if there actually is no efect Wee thus
need to compare what we found (the t-tests to what we would expect if there is no actual
diference.
On SPSS:
SPPS analyse Compare means Independent-samples T Test
Weith more degrees of freedom (bigger sample siies the t-distributon approaches to normal
distributon and makes higher t-values more strongly signifcant
Assumptons in normal t-test &o ANOVA
- Data is independent (between and within groupss
- DV at least interval level
- Variance in the conditons is roughly equal
- Data is normally distributed
If your sample siie is large enough there is ofen not a problem. If you have binary data, it is
a problem. E.g. binary data: I bought a product or not. (yes or nos.
How to create a good manipulaton
- Test if it works
o Do a pre-test
o Add a manipulaton check
- Test if it does not do something else
o Add a confound check
