Summary
Summary Chapter3-Econometrics-Multiple Linear Regression Model
- Course
- Institution
Chapter3-Econometrics-Multiple Linear Regression Model
[Show more]
Seller
Follow
partwi085
Content preview
Chapter 3Multiple Linear Regression Model
We consider the problem of regression when the study variable depends on more than one explanatory or
independent variables, called a multiple linear regression model. This model generalizes the simple linear
regression in two ways. It allows the mean function E ( y ) to depend on more than one explanatory variables
and to have shapes other than straight lines, although it does not allow for arbitrary shapes.
The linear model:
Let y denotes the dependent (or study) variable that is linearly related to k independent (or explanatory)
variables X 1 , X 2 ,..., X k through the parameters 1 , 2 ,..., k and we write
y X 11 X 2 2 ... X k k .
This is called the multiple linear regression model. The parameters 1 , 2 ,..., k are the regression
coefficients associated with X 1 , X 2 ,..., X k respectively and is the random error component reflecting the
difference between the observed and fitted linear relationship. There can be various reasons for such
difference, e.g., the joint effect of those variables not included in the model, random factors which can not
be accounted for in the model etc.
Note that the j th regression coefficient j represents the expected change in y per unit change in the j th
independent variable X j . Assuming E ( ) 0,
E ( y )
j .
X j
Linear model:
y E ( y )
A model is said to be linear when it is linear in parameters. In such a case (or equivalently )
j j
should not depend on any ' s . For example,
i) y 0 1 X is a linear model as it is linear in the parameters.
ii) y 0 X 1 can be written as
log y log 0 1 log X
y* 0* 1 x*
which is linear in the parameter 0* and 1 , but nonlinear is variables y* log y, x* log x. So it is
a linear model.
Econometrics | Chapter 3 | Multiple Linear Regression Model | Shalabh, IIT Kanpur
1
,iii) y 0 1 X 2 X 2
is linear in parameters 0 , 1 and 2 but it is nonlinear is variables X . So it is a linear model
1
iv) y 0
X 2
is nonlinear in the parameters and variables both. So it is a nonlinear model.
v) y 0 1 X 2
is nonlinear in the parameters and variables both. So it is a nonlinear model.
vi) y 0 1 X 2 X 2 3 X 3
is a cubic polynomial model which can be written as
y 0 1 X 2 X 2 3 X 3
which is linear in the parameters 0 , 1 , 2 , 3 and linear in the variables X 1 X , X 2 X 2 , X 3 X 3 .
So it is a linear model.
Example:
The income and education of a person are related. It is expected that, on average, a higher level of education
provides higher income. So a simple linear regression model can be expressed as
income 0 1 education .
Not that 1 reflects the change in income with respect to per unit change in education and 0 reflects the
income when education is zero as it is expected that even an illiterate person can also have some income.
Further, this model neglects that most people have higher income when they are older than when they are
young, regardless of education. So 1 will over-state the marginal impact of education. If age and education
are positively correlated, then the regression model will associate all the observed increase in income with an
increase in education. So a better model is
income 0 1 education 2 age .
Often it is observed that the income tends to rise less rapidly in the later earning years than is early years. To
accommodate such a possibility, we might extend the model to
income 0 1education 2 age 3age 2
This is how we proceed for regression modeling in real-life situation. One needs to consider the experimental
condition and the phenomenon before making the decision on how many, why and how to choose the
dependent and independent variables.
Econometrics | Chapter 3 | Multiple Linear Regression Model | Shalabh, IIT Kanpur
2
,Model set up:
Let an experiment be conducted n times, and the data is obtained as follows:
Observation number Response Explanatory variables
y X1 X 2 X k
1 y1 x11 x12 x1k
2 y2 x21 x22 x2 k
yn xn1 xn 2 xnk
n
Assuming that the model is
y 0 1 X 1 2 X 2 ... k X k ,
the n-tuples of observations are also assumed to follow the same model. Thus they satisfy
y1 0 1 x11 2 x12 ... k x1k 1
y2 0 1 x21 2 x22 ... k x2 k 2
yn 0 1 xn1 2 xn 2 ... k xnk n .
These n equations can be written as
y1 1 x11 x12 x1k 0 1
y2 1 x21 x22 x2 k 1 2
yn 1 xn1 xn 2 xnk k n
or y X .
In general, the model with k explanatory variables can be expressed as
y X
where y ( y1 , y2 ,..., yn ) ' is a n 1 vector of n observation on study variable,
x11 x12 x1k
x21 x22 x2 k
X
xn1 xn 2 xnk
is a n k matrix of n observations on each of the k explanatory variables, ( 1 , 2 ,..., k ) ' is a k 1
vector of regression coefficients and (1 , 2 ,..., n ) ' is a n 1 vector of random error components or
disturbance term.
If intercept term is present, take first column of X to be (1,1,…,1)’.
Econometrics | Chapter 3 | Multiple Linear Regression Model | Shalabh, IIT Kanpur
3
, Assumptions in multiple linear regression model
Some assumptions are needed in the model y X for drawing the statistical inferences. The following
assumptions are made:
(i) E ( ) 0
(ii) E ( ') 2 I n
(iii) Rank ( X ) k
(iv) X is a non-stochastic matrix
(v) ~ N (0, 2 I n ) .
These assumptions are used to study the statistical properties of the estimator of regression coefficients. The
following assumption is required to study, particularly the large sample properties of the estimators.
X 'X
(vi) lim exists and is a non-stochastic and nonsingular matrix (with finite elements).
n
n
The explanatory variables can also be stochastic in some cases. We assume that X is non-stochastic unless
stated separately.
We consider the problems of estimation and testing of hypothesis on regression coefficient vector under the
stated assumption.
Estimation of parameters:
A general procedure for the estimation of regression coefficient vector is to minimize
n n
M ( i ) M ( yi xi11 xi 2 2 ... xik k )
i 1 i 1
for a suitably chosen function M .
Some examples of choice of M are
M ( x) x
M ( x) x 2
M ( x) x , in general.
p
We consider the principle of least square which is related to M ( x) x 2 and method of maximum likelihood
estimation for the estimation of parameters.
Econometrics | Chapter 3 | Multiple Linear Regression Model | Shalabh, IIT Kanpur
4
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller partwi085. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $4.49. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
75323 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now