Statistics for Experimental Research Summary for 2017 Final Exam p. 1
Part 1: Statistical Terms, Formulas and Definitions update 23 mei Tahrim Ramdjan
Term Definition Sym Formula
Dependent /
outcome variable The variable that is measured and on which you (hope to) monitor an effect.
Field Ch. 1
Independent /
predictor variable The variable that is controlled, manipulated or chosen, with potential causal impact.
/ factor The independent is also known as ‘factor’ with multiple ‘levels’ (=conditions).
Field Ch. 1
An experimental design is characterized by two components:
Experimental
1) Random assignment to conditions
design
2) The presence of a control condition
Field Ch. 1
The goal is to study effects in isolation, to draw conclusions about causality.
Between
subjects variable Every participant gets a different manipulation.
Field Ch. 1
Within subjects
Every participant gets all different manipulation.
variable
e.g. measuring at different times.
Field Ch. 1
Factorial design
A fully factorial design includes only between subjects variables.
Field Ch. 1
Mixed design A mixed design includes both between subjects variables and within subjects
Field Ch. 1 variables.
Variation Systematic: due to experimental manipulation or confounds.
Field Ch. 1 Unsystematic: due to random factors.
Categorical Binary: only two categories (e.g. male/female, dead/alive)
variables Nominal: more than two categories (e.g. vegetarian, vegan, omnivore, …)
Field Ch. 1 Ordinal: categories have a logical order (e.g. fail, pass, honors)
Continuous Interval: equal intervals represent equal differences (e.g. temperature)
variables Ratio: with absolute zero; ratios of scores make sense (e.g. reaction time, distance)
Field Ch. 1 Likert scales, also other rating and sliding scales
Normal
Distribution of data (with values of observations on x-axis, and frequency on the y-axis)
distribution
that is bell-shaped and symmetrical
Field Ch. 1
Skewness The degree to which most frequent scores are clustered in one tail (end) of the
Field Ch. 1 distribution; indicates asymmetry from the normal distribution
Kurtosis The degree to which scores are clustered in the tails and to which the peaks are high in
Field Ch. 1 the distribution.
Mode
Central tendency: The most frequent score.
Field Ch. 1
Central tendency: The middle score, when all scores are ordered from smallest to
Median greatest.
Field Ch. 1 à Got an even amount of numbers? Find the middle pair of numbers, and find the
value halfway between them.
Mean / average
Central tendency: Sum of scores, divided by number of scores.
Field Ch. 1
A score very different from the rest of the data. Outliers may bias the parameters. You
Outlier
can detect outliers by:
Field Ch. 1
à subjectively inspecting the data matrix (discouraged!)
,Statistics for Experimental Research Summary for 2017 Final Exam p. 2
Part 1: Statistical Terms, Formulas and Definitions update 23 mei Tahrim Ramdjan
à boxplot (small dot or star)
à transformation to Z-scores (see part 2 of this summary)
Range
Dispersion: Difference between smallest and largest score.
Field Ch. 1
Quartiles Dispersion: Three values that split the data into four equal parts (that all make up 25%
Field Ch. 1 of the frequencies).
Deviance
Dispersion: Difference between score and mean score.
Field Ch. 1
Sum of squared
Dispersion: Total deviance of scores from the mean (squared to make up for all the
errors
negative errors).
Field Ch. 1
Variance Dispersion: Average deviance of scores from the mean: average error between mean
Field Ch. 1 and observations.
Standard
deviation Dispersion: Square root of the variance. -
Field Ch. 1
Standardized manner to indicate the probability of a particular
Z-scores value, using the normal distribution. 𝑋−𝜇
Field Ch. 1 à 95.0% of data should fall within Z -1.96 and Z 1.96.
z 𝑠
à 99.9% of data should fall within Z -3.09 and Z 3.09.
Model fit
The extent to which a statistical model represents the data well.
Field Ch. 2
General Linear
Most statistical analyses in this course are based on the same linear model, 𝑏𝑋&
Model + 𝑒𝑟𝑟𝑜𝑟&
fitting the data with a straight line.
Field Ch. 2
Sampling
distribution The frequency distribution of different sample means, from the same population.
Field Ch. 2
Central limit Prescribes that in large samples with n > 30, the sampling distribution of the sample
theorem becomes approximately normal with a mean equal to the population mean and
.
Field Ch. 2 standard deviation = 𝜎 = .
/
Standard deviation of sample means.
à How does it differ from standard deviation?
SD: the extent to which individuals within the sample differ from the sample mean.
Standard error of
SE: the extent to which the means of different samples differ from the population
the mean
mean.
Field Ch. 2
à How to interpret?
Large SE: much variability à sample may not be representative.
Small SE: little variability à sample likely to be representative.
Boundaries around a parameter (like the mean) within which the true value of this
parameter in the population probably would fall. The interval has the sample value as
midpoint; an upper limit; and a lower limit. The limits are calculated, based on the
(95%) confidence
normal distribution. (Constructed in such a way that in 95% of the samples, the true
intervals value falls within these limits)
Field Ch. 2
à How to interpret?
Wide interval: sample may not be representative.
Narrow interval: sample likely to be representative.
,Statistics for Experimental Research Summary for 2017 Final Exam p. 3
Part 1: Statistical Terms, Formulas and Definitions update 23 mei Tahrim Ramdjan
Null hypothesis H1: alternative hypothesis: there is a relation between variables. Proven when p-value
significance ≤ 0.05.
testing (NHST) H0: null-hypothesis: there is no relation between the variables.
Field Ch. 2 à Basic principle of NHST: Fit a model that represents the alternative hypothesis to
the data, then evaluate the probability of the model given the null hypothesis.
False positive. Saying the study did show an effect while in
Type I error fact there is no effect.
Field Ch. 2 *symbol indicates probability of Type I error
𝛼*
à Also 𝛼: level of significance.
False negative. Saying the study did not show an effect while
Type II error
Field Ch. 2
in fact there is an effect. 𝛽*
*symbol indicates probability of Type II error
Probability of obtaining a result equal or more extreme than
what was observed, if the null hypothesis were true. P-value
usually has to be 0.05 for an effect to be significant.
p-value
Field Ch. 2
à The p-value does not indicate effect size or importance. p
à For a p-value below 0.05, we can say that the model would
be unlikely to fit this well if there was no effect. Therefore, we
can reject the null-hypothesis.
Bonferroni The more independent comparisons you make, the more likely you are to 𝛼
correction find at least one p-value below 0.05. Therefore, you need to adjust the p- 𝐾
k = number
Field Ch. 2 value with the number of tests. This is the Bonferroni correction. of tests
The ability of a test to find a genuine effect: to say that there is an effect,
when there actually is one. You want the power to be as high as possible.
à calculation: 1 – probability of type II (false negative) error
Power à Power depends on: 1− 𝛽
Field Ch. 2
- The effect size
- The level of significance
- The sample size
Estimate of effect size. Can be used to compare across
different studies. Difference between two means, divided by
(pooled) standard deviation.
Cohen’s d 𝜇5 − 𝜇6
Field Ch. 2
à How to interpret? d 𝑠
d = .2 à small effect
d = .5 à medium effect
d = .8 à large effect
Histogram Graph: Plotting a score (x-axis) and corresponding frequency (y-axis). Helps to identify
Field Ch. 3 & 4 shape of the distribution and unusual scores.
Bar chart Graph: Used to display means (mostly); can display data for different groups. Makes
Field Ch. 3 & 4 use of error bars, that indicate either the confidence interval, SD or SE.
Graph: Displays the range of data, showing median and quartiles. Shows the limits of
Boxplot
the middle 50% of scores (interquartile range). Outliers can be spotted by stars or
Field Ch. 3 & 4
small dots.
Scatterplot Graph: Used to display relationships between variables. Each dot represents a
Field Ch. 3 & 4 subjects score on one variable, against their score on another variable.
Assumption of Assumption: Observations in a study should be independent from one another.
independence à When is this assumption violated?
Field Ch. 5 If the errors in a model appear to be related to each other.
, Statistics for Experimental Research Summary for 2017 Final Exam p. 4
Part 1: Statistical Terms, Formulas and Definitions update 23 mei Tahrim Ramdjan
Assumption of Assumption: Normal distribution of data.
normality à When is this assumption violated?
Field Ch. 5 If you create a PP-plot or QQ-plot and the expected z-scores do not coincide with the
actual z-scores of the data (the data will not appear to be on a diagonal line).
à How to prevent?
Central limit theorem: use larger sample sizes; run at least 30 participants per
condition.
à Use: Kolmogorov-Smirnov & Shapiro-Wilk test (see part 2 of this summary)
Assumption: Each of the different groups has the same variance.
Assumption of
à When is this assumption violated?
homogeneity of
When different groups have different variances in your model.
variances / à How to prevent?
homoscedasticity
Make sure you have equal group sizes.
Field Ch. 5
à Use: Levene’s test (see part 2 of this summary)
Standardized scale that indicates to what extent there is a
relationship between two variables.
Pearson’s à Based on covariance: deviations from mean in one variable,
correlation meet deviations from mean in other variable.
à Always lies between -1 and 1. You test whether the
r
coefficient
Field Ch. 7 correlation coefficient significantly differs from 0.
à Correlation coefficients indicate also the size of the
correlation!
Measures how well the regression model fits the data: how
R2 𝑆𝑆8
Field Ch. 7
much variance in the outcome can be explained by the model. R2 𝑆𝑆9
à You want R2 to be as high as possible!
Indicates whether different items in a scale (like a
questionnaire), reliably measure the same construct.
à Idea: Individual items should produce consistent with the
Cronbach’s alpha whole questionnaire.
Field Ch. 7 à It is a correlation in the sense of calculating people’s scores
𝛼
on one subset of variables, against people’s scores on another
subset.
à Reliable above .7
Predicting the value from one variable (outcome variable) from another
variable (predictor variable). The predictors can be either interval,
continuous or categorical variables. 𝑏: + 𝑏5 𝑋5
à The regression line is merely a model and doesn’t have to reflect reality. + 𝜀&
Simple b0 =
à We test how well this model fits the observed data, looking at deviations intercept
regression
between what the model predicts and what the actual data says. b1 =
Field Ch. 8 regression
à Follow-up coefficient of
1. Interpret the fit of the model with the F-value and its according p-value. the predictor
2. Assess the predictors with a t-test, indicating whether the regression
coefficient significantly differs from 0.
Error between values predicted by model, and actual value of observation. Examining
residuals can inform about outliers.
Residuals à Standardized residuals are residuals transformed to z-scores:
Field Ch. 8
Max. 1% of standardized residuals should be Z>2.58 or Z<-2.58
No standardized residuals should be Z>3.29 or Z<-3.29
