Total summary W1 – W7. Weekly tests, Weekly assignments, Lecture.
EBCP-1: How can we reduce the gap between research and
practice.
Independent samples t-test
o T-statistic
− y (w a l k ) − − y ( s i t ) but can also be rewritten as
s e [ − y ( w a l k )− − y ( s i t ) ]
an regression analysis.
o When to use independent sample t-test?
If you want to compare two groups e.g. intervention vs control, than you need to measure
the difference between the groups by comparing the means of both groups weighted by the
variability (standard error) of that difference.
o Hypothesis testing
H0: − y ( w a l k )=− y ( sit ) . You always test an null hypothesis.
If the data are consistent with H0, then we cannot reject H0. Intervention has failed.
If the data are inconsistent with H0, then we reject H0. Intervention is effective =
significant. i.e., data are ‘too unusual’ compared to what we would expect if H 0 were
true. The data are not inconsistent with HA.
However this does not imply that the data provides evidence for the
alternative hypothesis HA.
o Example
Post-measurement(i) = b0 + b1* Treatment(i)
y(i) = b0 + b1* x(i).
The dependent variable is an linear function of the independent variable plus
an intercept.
So B1 is weighted by an specific factor and then you add an intercept to it.
It’s the basis of most of the statistics.
o B1 is what you are interested in with hypothesis testing!
The absolute value of B1 parameter is equal to the difference in average number of
errors between two groups.
Interpretation based on B1: The control group made more errors in the post-
measurement (M=6.83, SD=1.27) than the intervention group (M=4.5, SD=90). This
difference was significant (t(22)=5.19, pf<.001).
o Variability distributions (narrow vs broad)
Red line shows a small difference /little variability meaning that the means are almost the
same. If the means are almost the same, you can trust the observed difference and can give it
a high weight.
Green line shows, this is not something what you
want, because this line shows a lot of variability in the score. Here, the
observed difference cannot be trusted and therefore we weight this
observed difference by variability in that difference (standard error).
o Example independent sample test with output
,There is a treatment effect, in cognitive therapy (M = . 523) one scores significantly higher than
without therapy (M = -. 393); t(30)= -3.094, p =. 004). This means that there is a significant
difference between the two groups. Since, the t-statistic is negative, this means that the wellbeing
decreases of participants in the ‘waitlist’ group.
o Example independent sample t-test
Suppose you want to know whether proximity seeking differs between boys and girls.
à Discrete independent variable: Sex
à Continuous dependent variable: Proximity seeking
à Effect size: d-family (e.g., Cohen’s d or Hedges g), as the independent variable is
nominal/binary/dichotomous/ discreet and dependent variable is continuous.
o Example independent sample t test (change in baseline).
Assume the score of adolescents on a social skills test depends on the number of months they
have been in a nursery as a 0-4 year old. In a research group the "months in nursery" varies
between 0 and 40.
The regression model is: y(i)=b0+b1*x(i).
b0 = 4,
b1 = 0.2. This is the increase in social skills score (y) per increment
of one-month nursery (x). Thus, if the number of "months in nursery" increases by 1 month, the
social competence score increases by 0.2.
Ordinary Least Squares model: Yi = b0 + b1 * Xi + ri
Yi = value of dependent variable for participant i
Xi = observed value of independent variable
b0 = intercept (= predicted value of Y when X = equal to 0, zero =
centered). The substantive meaning of the intercept therefore depends on
scaling of the independent variable.
b1 = slope (= predicted change in Y per 1-unit increase in X)
ri = error (= difference between the predicted value of Y and the actual
value of Y for a given case)
Interpretation of b0 explained
o The intercept (b0) gives the baseline social skills score (y) at zero months at the nursery.
Centered!!
Interpretation intercept/ b0: This means that when the participant has never been in nursery
(0 months), the score of this adolescent on the social skills test would be 4.
o The social skills score (y) at zero months in nursery (x=0).
Interpretation slope/b1: This means that the score of adolescents on a social skills test will
increase with 0.2 whenever they have been a month longer in nursery.This is the increase
in social skills score (y) per increment of one-month nursery (x). Thus, if the number of
"months in nursery" increases by 1 month, the social competence score increases by 0.2.
, Shift baseline to the average: The baseline has shifts when the researcher subtract the
average number of months at the nursery for each individual. So from zero month to the
average number of months.
New interpretation of b0/ intercept: the baseline social skills score (y) at the average
number of months at the nursery
Assumptions independent sample t-test (same as one-way ANOVA)
1) Dependent variable are continuous.
The first assumption, continuity is violated when the outcome variable is nominal
(e.g. nominal, whether or not a reading test has been passed).
2) Within the two groups, the dependent variables are normally distributed,
The second assumption, normality is violated when the t-distribution in SPSS shows an bimodal
(two tops) figure instead of an nominal (one top) figure.
(e.g. binominal; within the remedial teaching group half has progressed considerably
whereas the other half hasn’t; then the score is bimodal and shows an bimodal figure).
3) Variance in the dependent variable in one group is equal to variance in the dependent
variable in the other group (if not, check the outcomes for unequal variances)
The third assumption, homoscedasticity OR equal variance is violated when you see an output in
SPSS where the standard deviations of the two groups are unequal. Or, you can check this with
the Levene’s test which needs to be insignificant P > .05.
(e.g., there are two classes. Children in class 1 all received a new teaching method and had
grades of 9 and 10, meaning that the variance between the grades 9 and 10 is small.
Children in class 2 all received an old teaching method and their grades varied between a 4
and a 10, meaning that the variance between having a 4 and having a 10 is big). If you are
comparing class 1 and class 2 grades, you will immediately notice that the variances within
the two groups are unequal, which is not something you want to measure).
4) One of the most important: participants are independent of one another
The fourth assumption, independence is violated when participants are related in some way, which
is known as nested data/ nesting. The problem of nested data is that the results in a t-test become
too liberal and the conclusions from the results become invalid.
Definition Nested data: data where individuals share a common ground. E.g. siblings that
share the same upbringing, clients in a nursing home that all received the care of the same
group of professionals.
Definition liberal results: Liberal results are results that are significant whereas they
wouldn’t be significant at all! A big implication is on the decision making part of the clinician
if they are not aware of these liberal results.
For example, based on a t-test you have compared an intervention group to a control
group. Results shows that the intervention group scores significantly better as
compared to the control group. However, in reality the intervention group is not score
significantly better. Based on the ‘wrong results’ a researcher/ clinician can then
make a wrong decision by implementing an ‘effective’ intervention within their
practice, whereas in reality it is possibly not (that) effective at all.
Definition AB design not possible here!
An AB design implies that the same subjects go through two phases, for example first the waiting
list in phase A, and then the intervention in phase B. If this is the case, you cannot perform an
independent-samples t-test on the data since the observations are not independent, and one of
the assumptions for this independent t test is to have independent variables.
Regression analysis = One-way ANOVA.
o When to use an regression analysis?
, When you want to fit a line y(i) = b0 + b1*x(i) through the data (scatterplot points) to predict
the value of one variable from the value of another variable.
x (the predictor variable) is used to predict y (the regressor variable)
the values of x (the predictor) variable are fixed.
y (the regressor) is measured for all the values of x.
The regression line should minimize the deviation in y from that line.
Thus, there should be a low variance in the residuals in order to have a
good fit of the line true the data points.
o Residuals
Display the scatterplot/variance above and belove the line and the residuals .
It shows the difference between measured Y and predicted Y^ .
The variance in the residuals tells us something about the spread of the scatter
points and indicate the fit of the line.
So if there is a big residual variance, then it may fits the data, but the line itself
fits the data very poorly.
The goodness of fit not very strong if there is a big residual variance.
The variance explained is not very high if the residual variance is big.
o The regression model (F-value, t-value and p-value)
T-statistic
b 1−β 0
t=
S Eb
T & P value: In order to know if B1 says something, you look at the t-value and p-
value.
For example, if the regression coefficient B1=-2.33 you will need to
have a t value that is significant. Given t(22) = -5.19, p<0.001, it indeed
looks like the regression coefficient deviates significantly from zero.
F-value: F(1,22) =26.950, p<.001. Treatment (x) is a significant predictor for the
post-measurement (y), In this case, it explains a significant proportion of the
variance.
o Hypothesis testing based on b1
You test the hypothesis about the slope of b1 (in most cases you test the null hypothesis).
The slope is an important description of the best fitting regression line and you also want to
have the ‘best fit’. It reflects how strongly x and y co-vary.
if b1 ≈ 0: This is the degree when x
and y co-vary is negligible meaning
that the value of y does not depend on
the value of x.
if b1 ≠ 0: x and y co-vary significantly.
Treatment is a significant predictor for the post-
measurement, F(1,22) =26.950, p<.001. The value
of the regression coefficient B1=-2.33 and deviates
significantly from zero, t(22) = -5.19, p<0.001.
B1, 2.33, is equal to the difference in average number of errors between the two groups.
o SPSS regression analysis: You get the results by comparing the means for each group
with each other.
EBCP-1: How can we reduce the gap between research and
practice.
Independent samples t-test
o T-statistic
− y (w a l k ) − − y ( s i t ) but can also be rewritten as
s e [ − y ( w a l k )− − y ( s i t ) ]
an regression analysis.
o When to use independent sample t-test?
If you want to compare two groups e.g. intervention vs control, than you need to measure
the difference between the groups by comparing the means of both groups weighted by the
variability (standard error) of that difference.
o Hypothesis testing
H0: − y ( w a l k )=− y ( sit ) . You always test an null hypothesis.
If the data are consistent with H0, then we cannot reject H0. Intervention has failed.
If the data are inconsistent with H0, then we reject H0. Intervention is effective =
significant. i.e., data are ‘too unusual’ compared to what we would expect if H 0 were
true. The data are not inconsistent with HA.
However this does not imply that the data provides evidence for the
alternative hypothesis HA.
o Example
Post-measurement(i) = b0 + b1* Treatment(i)
y(i) = b0 + b1* x(i).
The dependent variable is an linear function of the independent variable plus
an intercept.
So B1 is weighted by an specific factor and then you add an intercept to it.
It’s the basis of most of the statistics.
o B1 is what you are interested in with hypothesis testing!
The absolute value of B1 parameter is equal to the difference in average number of
errors between two groups.
Interpretation based on B1: The control group made more errors in the post-
measurement (M=6.83, SD=1.27) than the intervention group (M=4.5, SD=90). This
difference was significant (t(22)=5.19, pf<.001).
o Variability distributions (narrow vs broad)
Red line shows a small difference /little variability meaning that the means are almost the
same. If the means are almost the same, you can trust the observed difference and can give it
a high weight.
Green line shows, this is not something what you
want, because this line shows a lot of variability in the score. Here, the
observed difference cannot be trusted and therefore we weight this
observed difference by variability in that difference (standard error).
o Example independent sample test with output
,There is a treatment effect, in cognitive therapy (M = . 523) one scores significantly higher than
without therapy (M = -. 393); t(30)= -3.094, p =. 004). This means that there is a significant
difference between the two groups. Since, the t-statistic is negative, this means that the wellbeing
decreases of participants in the ‘waitlist’ group.
o Example independent sample t-test
Suppose you want to know whether proximity seeking differs between boys and girls.
à Discrete independent variable: Sex
à Continuous dependent variable: Proximity seeking
à Effect size: d-family (e.g., Cohen’s d or Hedges g), as the independent variable is
nominal/binary/dichotomous/ discreet and dependent variable is continuous.
o Example independent sample t test (change in baseline).
Assume the score of adolescents on a social skills test depends on the number of months they
have been in a nursery as a 0-4 year old. In a research group the "months in nursery" varies
between 0 and 40.
The regression model is: y(i)=b0+b1*x(i).
b0 = 4,
b1 = 0.2. This is the increase in social skills score (y) per increment
of one-month nursery (x). Thus, if the number of "months in nursery" increases by 1 month, the
social competence score increases by 0.2.
Ordinary Least Squares model: Yi = b0 + b1 * Xi + ri
Yi = value of dependent variable for participant i
Xi = observed value of independent variable
b0 = intercept (= predicted value of Y when X = equal to 0, zero =
centered). The substantive meaning of the intercept therefore depends on
scaling of the independent variable.
b1 = slope (= predicted change in Y per 1-unit increase in X)
ri = error (= difference between the predicted value of Y and the actual
value of Y for a given case)
Interpretation of b0 explained
o The intercept (b0) gives the baseline social skills score (y) at zero months at the nursery.
Centered!!
Interpretation intercept/ b0: This means that when the participant has never been in nursery
(0 months), the score of this adolescent on the social skills test would be 4.
o The social skills score (y) at zero months in nursery (x=0).
Interpretation slope/b1: This means that the score of adolescents on a social skills test will
increase with 0.2 whenever they have been a month longer in nursery.This is the increase
in social skills score (y) per increment of one-month nursery (x). Thus, if the number of
"months in nursery" increases by 1 month, the social competence score increases by 0.2.
, Shift baseline to the average: The baseline has shifts when the researcher subtract the
average number of months at the nursery for each individual. So from zero month to the
average number of months.
New interpretation of b0/ intercept: the baseline social skills score (y) at the average
number of months at the nursery
Assumptions independent sample t-test (same as one-way ANOVA)
1) Dependent variable are continuous.
The first assumption, continuity is violated when the outcome variable is nominal
(e.g. nominal, whether or not a reading test has been passed).
2) Within the two groups, the dependent variables are normally distributed,
The second assumption, normality is violated when the t-distribution in SPSS shows an bimodal
(two tops) figure instead of an nominal (one top) figure.
(e.g. binominal; within the remedial teaching group half has progressed considerably
whereas the other half hasn’t; then the score is bimodal and shows an bimodal figure).
3) Variance in the dependent variable in one group is equal to variance in the dependent
variable in the other group (if not, check the outcomes for unequal variances)
The third assumption, homoscedasticity OR equal variance is violated when you see an output in
SPSS where the standard deviations of the two groups are unequal. Or, you can check this with
the Levene’s test which needs to be insignificant P > .05.
(e.g., there are two classes. Children in class 1 all received a new teaching method and had
grades of 9 and 10, meaning that the variance between the grades 9 and 10 is small.
Children in class 2 all received an old teaching method and their grades varied between a 4
and a 10, meaning that the variance between having a 4 and having a 10 is big). If you are
comparing class 1 and class 2 grades, you will immediately notice that the variances within
the two groups are unequal, which is not something you want to measure).
4) One of the most important: participants are independent of one another
The fourth assumption, independence is violated when participants are related in some way, which
is known as nested data/ nesting. The problem of nested data is that the results in a t-test become
too liberal and the conclusions from the results become invalid.
Definition Nested data: data where individuals share a common ground. E.g. siblings that
share the same upbringing, clients in a nursing home that all received the care of the same
group of professionals.
Definition liberal results: Liberal results are results that are significant whereas they
wouldn’t be significant at all! A big implication is on the decision making part of the clinician
if they are not aware of these liberal results.
For example, based on a t-test you have compared an intervention group to a control
group. Results shows that the intervention group scores significantly better as
compared to the control group. However, in reality the intervention group is not score
significantly better. Based on the ‘wrong results’ a researcher/ clinician can then
make a wrong decision by implementing an ‘effective’ intervention within their
practice, whereas in reality it is possibly not (that) effective at all.
Definition AB design not possible here!
An AB design implies that the same subjects go through two phases, for example first the waiting
list in phase A, and then the intervention in phase B. If this is the case, you cannot perform an
independent-samples t-test on the data since the observations are not independent, and one of
the assumptions for this independent t test is to have independent variables.
Regression analysis = One-way ANOVA.
o When to use an regression analysis?
, When you want to fit a line y(i) = b0 + b1*x(i) through the data (scatterplot points) to predict
the value of one variable from the value of another variable.
x (the predictor variable) is used to predict y (the regressor variable)
the values of x (the predictor) variable are fixed.
y (the regressor) is measured for all the values of x.
The regression line should minimize the deviation in y from that line.
Thus, there should be a low variance in the residuals in order to have a
good fit of the line true the data points.
o Residuals
Display the scatterplot/variance above and belove the line and the residuals .
It shows the difference between measured Y and predicted Y^ .
The variance in the residuals tells us something about the spread of the scatter
points and indicate the fit of the line.
So if there is a big residual variance, then it may fits the data, but the line itself
fits the data very poorly.
The goodness of fit not very strong if there is a big residual variance.
The variance explained is not very high if the residual variance is big.
o The regression model (F-value, t-value and p-value)
T-statistic
b 1−β 0
t=
S Eb
T & P value: In order to know if B1 says something, you look at the t-value and p-
value.
For example, if the regression coefficient B1=-2.33 you will need to
have a t value that is significant. Given t(22) = -5.19, p<0.001, it indeed
looks like the regression coefficient deviates significantly from zero.
F-value: F(1,22) =26.950, p<.001. Treatment (x) is a significant predictor for the
post-measurement (y), In this case, it explains a significant proportion of the
variance.
o Hypothesis testing based on b1
You test the hypothesis about the slope of b1 (in most cases you test the null hypothesis).
The slope is an important description of the best fitting regression line and you also want to
have the ‘best fit’. It reflects how strongly x and y co-vary.
if b1 ≈ 0: This is the degree when x
and y co-vary is negligible meaning
that the value of y does not depend on
the value of x.
if b1 ≠ 0: x and y co-vary significantly.
Treatment is a significant predictor for the post-
measurement, F(1,22) =26.950, p<.001. The value
of the regression coefficient B1=-2.33 and deviates
significantly from zero, t(22) = -5.19, p<0.001.
B1, 2.33, is equal to the difference in average number of errors between the two groups.
o SPSS regression analysis: You get the results by comparing the means for each group
with each other.
