- Suppose that Y is a linear function of another variable X, with unknown parameters
beta1 and beta 2.
- Suppose that we have a sample of 4 observations with X values (as shown)
- If the relationship were an exact one, the 4 observations would lie on the straight
line (and we would have no trouble with obtaining the estimates of beta1 and
beta2). But in practice, this does not happen à the observations lie around the
straight line. à hence the values of Y are different than the values (of Y) on the
straight line
à To allow for such divergence, we will write the regression model as 𝑌 = 𝛽! + 𝛽" 𝑋 +
𝑢 , where u is the disturbance term
o Why does the disturbance term exist?
1. Omission of explanatory variables
2. Aggregation of variables
3. Measurement error
4. Model & functional misspecification
à Each value of Y exists of a nonrandom component, 𝛽! + 𝛽" 𝑋, and a random component
u.
Estimating a linear regression model
- We now use 𝑌( = 𝛽)! + 𝛽)" 𝑋, where 𝛽)! is an estimate of 𝛽! and 𝛽)" is an estimate of
𝛽" .
, - The line is called the fitted model and the values of Y predicted by it are called the
fitted values of Y (the R points).
- The discrepancies of the actual and the fitted values of Y are known as the residuals
(𝑢*).
Difference between u and 𝑢*
- This figure shows the disturbance term, u (difference between nonrandom
component and the actual observation).
, - This figure shows the residual, 𝑢* (difference between actual and fitted values).
Least Squares Criterion
Minimize RSS (residual sum of squares), where
$
𝑅𝑆𝑆 = / 𝑢*#" = 𝑢*!" + ⋯ + 𝑢*$"
#%!
à Draw the fitted line so as to minimize the sum of the squares of the residuals (RSS). à
Least squares criterion
- The aim is to fit the regression line: draw the fitted line so as to minimize sum of the
squared of residuals, RSS
- A least squares analysis begins with a set of data points plotted on a graph.
Independent variables are plotted on the horizontal x-axis while dependent variables
are plotted on the vertical y-axis. The analyst uses the least squares formula to
determine the most accurate straight line that will explain the relationship between
an independent variable and a dependent variable.
- We take the squares of the residuals because otherwise we would get a perfect fit
(horizontal line) à the sum of the residuals would be zero. To prevent this, we
should cancel out the negative values à this can be done by taking the squares of
the residuals.
Deriving Linear Regression Coefficients
1$ = 𝑌$ − 3
Calculating the residual: 𝑢 𝑌$
, à The values of Y1 (4), Y2 (3), Y3 (5) en Y4 (8), you can read from the Y axis.
à The values of 3𝑌$ (b1 and b2) we are going to estimate
Use partial derivatives:
- In order to calculate b1 and b2, set the equations equal to 0.
à 8b1 + 20b2 – 40 = 20b1 + 60b2 – 114 = 0
B1 = -3.333b2 + 6.166
à fill this in in the b2 formula
- We could name these values 𝐵 3! OLS and 𝐵
3" OLS to emphasize that these are the
particular values that satisfy the OLS criterion.
Derivation of Linear regression coefficients – General case with n observations
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller isabellaarnoldcrdenas. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $8.10. You're not tied to anything after your purchase.