Graded Homework 1
Solution Manual
Question 1. Choose the correct statement regarding the sum of
residuals calculated using Ordinary Least Squares (OLS).
Answer 1. (B) The sum of residuals will always be equal to zero if you include
intercept term in your model and you are using OLS to estimate...
graded homework 1 solution manual question 1 choose the correct statement regarding the sum of residuals calculated using ordinary least squares ols answer 1
Written for
Georgia Institute Of Technolog
Management
All documents for this subject (1)
Seller
Follow
Examhack
Reviews received
Content preview
Graded Homework 1
Solution Manual
Question 1. Choose the correct statement regarding the sum of
residuals calculated using Ordinary Least Squares (OLS).
Answer 1. (B) The sum of residuals will always be equal to zero if you include
intercept term in your model and you are using OLS to estimate the
coefficients.
When an intercept is included in multiple linear regression for a data set with
p points, the line is of the form:
In OLS (Ordinary Least Square) Regression, the objective is to minimize the
SSE (Sum of Squared Errors) to 0, which is obtained as follows:
And to find all the betas, we differentiate all the above equation against all of
betas individually including β0. We obtain ‘p+1’ equations to solve for all the
‘p+1’ betas. Differentiating against β0, we get:
The above equation is responsible for ensuring that the sum of residuals is 0.
Therefore, C and D are incorrect options.
We now must decide between A and B. One form of regression under
consideration is “Regression passing through Origin” (which is ideally not
considered to be True OLS) which is a special case. Absence of intercept still
means that the SSE is minimized, but it does not guarantee that SSE = 0. It
is only when the intercept is included that the residuals sum up to 0.
Thus, B is the correct option. One can think of the intercept as the catchall
term, which provides for shifting the line of regression closer to or away from
the X-axis (say the response terms are of the order 1000s, and thus the β0
value helps move the line of regression the said order, providing for finer
tuned remaining betas).
We understand that some ambiguity was observed regarding the
forms of regression. Here, the question was designed keeping in
mind the concept of the importance of the intercept term. While the
,true answer should be (B), we shall consider both (A) and (B) as the
right choice.
Question 2. Consider a simple linear regression model. Y ~ X. A fit
for this model will be a line in the X-Y plane. Now suppose we fit a
model for the opposite relationship: X ~ Y (that is, X dependent, and
Y independent). This new model yields a new regression line in the
X-Y plane. Which of the following statements are true about these
two lines?
Answer 2. (B) These lines will always intersect at the mean (Xmean, Ymean)
If we have that y is a dependent variable on x, (and for their respective
means ý and x́ ), then the least squares regression Y ~ X line is given
as:
CoV ( x , y )
( y− ý )= ( x− x́ )
Var ( x )
Similarly, the line for X ~ Y is given as:
CoV ( x , y )
(x−x́)= ( y− ý )
Var ( y )
Clearly, these two lines meet at the means ý and x́ .
The above equation is obtained using the least square estimator method for
estimating the values a and b while minimizing the residuals:
2
The final line is y=a+bx and we find a and b using equations obtained by
placing
∂ D(a , b) ∂ D(a ,b) CoV ( x , y )
= =0 which gives us a= ý−b x́ and b=
∂a ∂b Var ( x )
We can also write a small piece of code to check the same:
Residual standard error: 24.05 on 298 degrees of freedom
Multiple R-squared: 0.7218, Adjusted R-squared: 0.7209
F-statistic: 773.2 on 1 and 298 DF, p-value: < 2.2e-16
Residual standard error: 6.601 on 298 degrees of freedom
Multiple R-squared: 0.7218, Adjusted R-squared: 0.7209
F-statistic: 773.2 on 1 and 298 DF, p-value: < 2.2e-16
The graph produced is as follows (which meets at the mean of the Xs and Ys)
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller Examhack. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $15.49. You're not tied to anything after your purchase.
