Summary
Summary Statistics (FEB21007X)
- Course
- Institution
Comprehensive summary of Statistics (econometrics EUR)
[Show more]
Seller
Follow
LeonVerweij
Reviews received
Content preview
Week 1Random sampling assumption
Independent and identically distributed (i.i.d) random variables 𝑋! , … , 𝑋" with pdf 𝑓(𝑥; 𝜃),
where the parameter 𝜃 is unknown. 𝜃 lies in some parameter space Ω
Type of means
Population mean: the mean that is a characteristic of the type of the distribution
!
Sample mean: the observed mean, calculated as 𝑋+ = " ∑"#$! 𝑋#
Variables/value
The unknown/unobserved are random variables in uppercase, 𝑋! , … , 𝑋" ~(𝑖. 𝑖. 𝑑) 𝑓(𝑥; 𝜃)
The data consists of observed values in lower case from the random process, 𝑥! , … , 𝑥"
Estimates/estimate
An estimator 𝑇 = 𝑡(𝑋! , … , 𝑋" ) is a function of the random variables 𝑋! , … , 𝑋" , so also a rv
An estimate 𝑇 = 𝑡(𝑥! , … , 𝑥" ) is the observed value of the estimator 𝑇 (the function value) based
on the observed values 𝑥! , … , 𝑥"
Method of moments estimator (MME)
! %
Population moments: 𝐸6𝑋% 7 = 𝜇%& (𝜃! , … , 𝜃' ) and sample moments: " ∑"#$! 𝑋# = 𝑀%&
Obtain an estimator of the unknown parameter 𝜽 = (𝜃! , … , 𝜃' ) by equating population
moments to sample moments
Maximum likelihood estimator (MLE)
Use the value of the unknown parameter 𝜽 = (𝜃! , … , 𝜃' ) that is most likely to have generated
the observed data as estimate. The likelihood function is defined as 𝐿(𝜽) = 𝑓(𝑋! , … , 𝑋" ; 𝜽)
For a random sample 𝑋! , … , 𝑋" we have 𝐿(𝜽) = ∏"#$! 𝑓(𝑋# ; 𝜽)
The MLE 𝜽= = 6𝜃>! , … , 𝜃>' 7 is the value of 𝜽 that maximizes 𝐿(𝜽)
Maximum of 𝐿(𝜽)
(
Compute MLE as the solution of () 𝐿(𝜽) = 0, but it is not always possible to compute
(
∏"#$! 𝑓(𝑋# ; 𝜽). Taking the logarithm of 𝐿(𝜽) gives the log-likelihood ln6𝐿(𝜽)7, then compute
()
( (!
()
ln6𝐿(𝜽)7 = 0 and check if ()! ln6𝐿(𝜽)7 < 0. If 𝐿(𝜽) is not differentiable, mathematical
reasoning is required
Invariance properties
If 𝜃> is the MLE of 𝜃 and if 𝑢(𝜃) is a function of 𝜃, then 𝑢6𝜃>7 is an MLE of 𝑢(𝜃)
= = 6𝜃>! , … , 𝜃>' 7 denotes the MLE of 𝜽 = (𝜃! , … , 𝜃' ), then the MLE of
If 𝜽
𝜏(𝜽) = 6𝜏! (𝜽), … , 𝜏* (𝜽)7 is 𝜏6𝜽 =7 = E𝜏! 6𝜽
=7, … , 𝜏* 6𝜽
= 7F for 1 ≤ 𝑟 ≤ 𝑘
Unbiased estimator
An estimator 𝑇 is said to be an unbiased estimator of 𝜏(𝜃) if 𝐸(𝑇) = 𝜏(𝜃) for all 𝜃 ∈ Ω.
Otherwise, we say that 𝑇 is a biased estimator of 𝜏(𝜃)
The bias is defined as 𝐵𝑖𝑎𝑠(𝑇) = 𝐸(𝑇) − 𝜏(𝜃). With an unbiased estimator, on average the true
value is estimated (an accurate estimator)
Mean squared error (MSE)
+
If 𝑇 is an estimator of 𝜏(𝜃), then the MSE of 𝑇 is given by 𝑀𝑆𝐸(𝑇) = 𝐸6𝑇 − 𝜏(𝜃)7
+
Moreover, 𝑀𝑆𝐸(𝑇) = 𝑉(𝑇) + 6𝐵𝑖𝑎𝑠(𝑇)7 , so if 𝑇 is unbiased, the 𝑀𝑆𝐸(𝑇) = 𝑉(𝑇)
Uniformly minimum variance unbiased estimator (UMVUE)
An estimator 𝑇 ∗ of 𝜏(𝜃) is called a UMVUE of 𝜏(𝜃) is 𝑇 ∗ is unbiased for 𝜏(𝜃) and for any other
unbiased estimator 𝑇 of 𝜏(𝜃), 𝑉(𝑇 ∗ ) ≤ 𝑉(𝑇) for all 𝜃 ∈ Ω
Cramer-Rao lower bound (CRLB)
!
-." ())1
If 𝑇 is an unbiased estimator of 𝜏(𝜃), then the CRLB is 𝑉(𝑇) ≥ # ! or
"23 4567(8;)):;
#$
, !
-." ())1
𝑉(𝑇) ≥ #!
. If an unbiased estimator equals the CRLB, it is a UMVUE
<"2= ! 4567(8;)):>
#$
Relative efficiency
The relative efficiency of an unbiased estimator 𝑇 of 𝜏(𝜃) to another unbiased estimator 𝑇 ∗ of
?(@ ∗ )
𝜏(𝜃) is given by 𝑟𝑒(𝑇, 𝑇 ∗ ) = ?(@)
Efficiency
An unbiased estimator 𝑇 ∗ of 𝜏(𝜃) is said to be efficient if 𝑟𝑒(𝑇, 𝑇 ∗ ) ≤ 1 for all unbiased
estimators 𝑇 of 𝜏(𝜃), and all 𝜃 ∈ Ω. The efficiency of an unbiased estimator 𝑇 of 𝜏(𝜃) is given by
𝑒(𝑇) = 𝑟𝑒(𝑇, 𝑇 ∗ ) if 𝑇 ∗ is an efficient estimator of 𝜏(𝜃).
An unbiased estimator that reaches the CRLB is efficient and an efficient estimator is UMVUE
Estimator with several unknown parameters
If 𝑋! , … , 𝑋" have pdf 𝑓(𝑥; 𝜃! , … , 𝜃' ), solve the system of equations with 𝑘 equations.
! !
For MME, the system is: 𝐸(𝑋! ) = " ∑"#$! 𝑋#! , … , 𝐸(𝑋 ' ) = " ∑"#$! 𝑋#'
For MML, the system is all the 𝑘 partial derivatives of the log-likelihood equal to 0
Week 2
Simple consistency
Let {𝑇" } be a sequence of estimators of 𝜏(𝜃). These estimators are said to be consistent
estimators of 𝜏(𝜃) if for every 𝜀 > 0, lim 𝑃(|𝑇" − 𝜏(𝜃)| < 𝜀) = 1 for every 𝜃 ∈ Ω
"→B
C
This is equivalent to lim 𝑃(|𝑇" − 𝜏(𝜃)| > 𝜀) = 0. The notation is 𝑇" → 𝜏(𝜃)
"→B
A consistent estimator converges in probability to the true value
MSE consistency
If {𝑇" } be a sequence of estimators of 𝜏(𝜃), then they are called MSE consistent if
+
lim 𝐸6𝑇" − 𝜏(𝜃)7 = 0 for every 𝜃 ∈ Ω. MSE consistency ⟹ (simple) consistency
"→B
Asymptotically unbiased estimator
A sequence of estimators {𝑇" } is said to be asymptotically unbiased for 𝜏(𝜃) if lim 𝐸(𝑇" ) = 𝜏(𝜃)
"→B
for all 𝜃 ∈ Ω. Estimator is MSE consistent ⟺ it is asymptotically unbiased and lim 𝑉( 𝑇" ) = 0
"→B
Law of Large Numbers (LLN)
If 𝑋! , … , 𝑋" is a random sample from a distribution with finite mean and variance, then
! C
𝑋+" = " ∑"#$! 𝑋# → 𝐸(𝑋)
Continuous mapping theorem
C C
If 𝑌" → 𝑐, then for any function 𝑔(𝑦) that is continuous at 𝑐, 𝑔(𝑌" ) → 𝑔(𝑐)
This is also valid for 𝑘-dimensional vectors
Convergence in probability results theorem
C C
If 𝑋" and 𝑌" are two sequences of random variables such that 𝑋" → 𝑐 and 𝑌" → 𝑑, then:
C
- 𝑎𝑋" + 𝑏𝑌" → 𝑎𝑐 + 𝑏𝑑
C
- 𝑋" 𝑌" → 𝑐𝑑
C
- 𝑋" /𝑐 → 1 for 𝑐 ≠ 0
C
- 1/𝑋" → 1/𝑐 if 𝑐 ≠ 0
C
- g𝑋" → √𝑐 if 𝑐 > 0
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller LeonVerweij. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $7.52. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
52510 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now