Correlation and Regression Analysis

Correlation & Regression‬
‭

‭Definitions‬
‭➔‬‭correlation and regression are‬‭statistical techniques‬‭used to examine relationships between‬
‭variables‬
‭◆‬ ‭correlation‬‭: determines the strength of an association‬‭between two quantitative‬
‭variables‬
‭◆‬ ‭simple regression‬‭: predicts one quantitative dependent‬‭variable from an independent‬
‭variable‬
‭●‬ ‭dependent variable = y or the criterion‬
‭●‬ ‭independent variable = x or the predictor‬
‭◆‬ ‭multiple regression‬‭: predicts one criterion from multiple‬‭predictors‬

‭Pearson’s Product-Moment Correlation Coefficient (R)‬
‭➔‬‭r‬‭is the coefficient that represents the strength‬‭and the direction of the linear relationship‬
‭between two variables‬
‭◆‬ ‭correlation of a‬‭sample‬‭= “Pearson’s r” or just “r”‬
‭●‬ ‭lowercase “r” because uppercase R represents multiple regressions‬
‭◆‬ ‭correlation of a‬‭population‬‭= ⍴ (rho)‬
‭◆‬ ‭absolute value‬‭of r determines‬‭strength‬‭of the relationship‬‭between x and y‬
‭●‬ ‭the magnitude of the value (regardless of positive or negative)‬
‭●‬ ‭r = 0; no correlation‬
‭●‬ ‭r = -1 or 1; perfect correlation‬
‭○‬ ‭very rare/highly unlikely‬
‭○‬ ‭means x can perfectly predict y‬
‭◆‬ ‭the‬‭sign‬‭(positive or negative) determines the‬‭direction‬‭of the relationship‬
‭●‬ ‭- value = negative relationship‬
‭●‬ ‭+ value = positive relationship‬
‭➔‬‭the values for r or rho are‬‭always‬‭between‬‭-1 and‬‭1‬
‭➔‬‭think of r as a way of looking at‬‭how closely the‬‭data clusters around the regression line‬
‭◆‬ ‭scatterplots are useful for this (‬‭review of scatterplots‬‭in the slides from lecture‬‭)‬

‭Testing A Correlation‬
‭➔‬‭to determine whether correlation signifies a sampling error or an actual relationship existing,‬
‭you would have to run a‬‭t-test‬

, ‭➔‬‭first step‬‭= state the hypothesis and the degrees of freedom‬
‭◆‬ ‭for two-tailed test‬
‭●‬ ‭H‬‭0‬ ‭= no correlation between the 2 variables (p=0)‬
‭●‬ ‭H‬‭A‬ ‭= there is correlation between the 2 variables‬‭(p≠0)‬
‭◆‬ ‭for one-tailed test‬
‭●‬ ‭state the directionality of the correlation‬
‭◆‬ ‭DF‬‭= n - 2‬
‭●‬ ‭n = the number of points of x and y together‬
‭➔‬‭second step‬‭= state the assumptions‬
‭◆‬ ‭the DV and the IV are both normally distributed‬
‭◆‬ ‭there are no outliers in either the DV or the IV; no bivariate outliers‬
‭●‬ ‭bivariate outliers = outliers when considering both the variables together‬
‭●‬ ‭correlations are not resistant to outliers, especially when the‬‭n‬‭is small‬
‭◆‬ ‭the DV and IV are linearly related‬
‭●‬ ‭correlations only capture linearity‬
‭●‬ ‭no way to actually test this, would just have to look at a scatterplot‬
‭○‬ ‭hence for this class, just state that it’s assumed‬
‭◆‬ ‭the correlation between the two variables must be significant to run the simple‬
‭regression‬
‭●‬ ‭regression equation is only calculated if the correlation is significant, in other‬
‭words if the null is rejected‬
‭➔‬‭third step‬‭= find Pearson’s‬‭r‬
‭➔‬‭fourth step‬‭= test the correlation using t-critical‬
‭➔‬‭fifth step‬‭= if the null is rejected (aka significance‬‭is found) calculate the regression equation‬

‭Effect Size for R‬
‭➔‬‭effect size for r = r‬‭2‬
‭➔‬‭similar to Cohen’s‬‭d‬
‭◆‬ ‭d‬‭tests the magnitude of the difference of the two‬‭variables‬
‭◆‬ ‭r‬‭tests the magnitude of the relationship of the two‬‭variables‬
‭●‬ ‭the proportion of variance explained‬
‭➔‬‭r‬ ‭explains‬‭how much of the variance‬‭in one of the‬‭variables(y) can be explained by the‬
‭2‬


‭relationship of it with the other variable (x), the‬‭rest being attributed to error‬
‭◆‬ ‭i.e. r‬‭2‬ ‭= 0.9025; hence 90.25% of the variance in‬‭weight can be explained by the‬
‭relationship between age and weight, the rest being attributed to by error‬
‭◆‬ ‭basically looking at the overlap between variable x and variable y‬