Samenvatting
Samenvatting Statistiek (FEB21007)
- Instelling
- Erasmus Universiteit Rotterdam (EUR)
Uitgebreide samenvatting van Statistiek (econometrie EUR)
[Meer zien]Voorbeeld 2 van de 8 pagina's
Voorbeeld 2 van de 8 pagina's
In winkelwagengesponsord bericht van onze partner
Voorbeeld van de inhoud
Week 1Random sampling assumption
Independent and identically distributed (i.i.d) random variables 𝑋! , … , 𝑋" with pdf 𝑓(𝑥; 𝜃),
where the parameter 𝜃 is unknown. 𝜃 lies in some parameter space Ω
Type of means
Population mean: the mean that is a characteristic of the type of the distribution
!
Sample mean: the observed mean, calculated as 𝑋+ = " ∑"#$! 𝑋#
Variables/value
The unknown/unobserved are random variables in uppercase, 𝑋! , … , 𝑋" ~(𝑖. 𝑖. 𝑑) 𝑓(𝑥; 𝜃)
The data consists of observed values in lower case from the random process, 𝑥! , … , 𝑥"
Estimates/estimate
An estimator 𝑇 = 𝑡(𝑋! , … , 𝑋" ) is a function of the random variables 𝑋! , … , 𝑋" , so also a rv
An estimate 𝑇 = 𝑡(𝑥! , … , 𝑥" ) is the observed value of the estimator 𝑇 (the function value) based
on the observed values 𝑥! , … , 𝑥"
Method of moments estimator (MME)
! %
Population moments: 𝐸6𝑋% 7 = 𝜇%& (𝜃! , … , 𝜃' ) and sample moments: " ∑"#$! 𝑋# = 𝑀%&
Obtain an estimator of the unknown parameter 𝜽 = (𝜃! , … , 𝜃' ) by equating population
moments to sample moments
Maximum likelihood estimator (MLE)
Use the value of the unknown parameter 𝜽 = (𝜃! , … , 𝜃' ) that is most likely to have generated
the observed data as estimate. The likelihood function is defined as 𝐿(𝜽) = 𝑓(𝑋! , … , 𝑋" ; 𝜽)
For a random sample 𝑋! , … , 𝑋" we have 𝐿(𝜽) = ∏"#$! 𝑓(𝑋# ; 𝜽)
The MLE 𝜽= = 6𝜃>! , … , 𝜃>' 7 is the value of 𝜽 that maximizes 𝐿(𝜽)
Maximum of 𝐿(𝜽)
(
Compute MLE as the solution of () 𝐿(𝜽) = 0, but it is not always possible to compute
(
∏"#$! 𝑓(𝑋# ; 𝜽). Taking the logarithm of 𝐿(𝜽) gives the log-likelihood ln6𝐿(𝜽)7, then compute
()
( (!
()
ln6𝐿(𝜽)7 = 0 and check if ()! ln6𝐿(𝜽)7 < 0. If 𝐿(𝜽) is not differentiable, mathematical
reasoning is required
Invariance properties
If 𝜃> is the MLE of 𝜃 and if 𝑢(𝜃) is a function of 𝜃, then 𝑢6𝜃>7 is an MLE of 𝑢(𝜃)
= = 6𝜃>! , … , 𝜃>' 7 denotes the MLE of 𝜽 = (𝜃! , … , 𝜃' ), then the MLE of
If 𝜽
𝜏(𝜽) = 6𝜏! (𝜽), … , 𝜏* (𝜽)7 is 𝜏6𝜽 =7 = E𝜏! 6𝜽
=7, … , 𝜏* 6𝜽
= 7F for 1 ≤ 𝑟 ≤ 𝑘
Unbiased estimator
An estimator 𝑇 is said to be an unbiased estimator of 𝜏(𝜃) if 𝐸(𝑇) = 𝜏(𝜃) for all 𝜃 ∈ Ω.
Otherwise, we say that 𝑇 is a biased estimator of 𝜏(𝜃)
The bias is defined as 𝐵𝑖𝑎𝑠(𝑇) = 𝐸(𝑇) − 𝜏(𝜃). With an unbiased estimator, on average the true
value is estimated (an accurate estimator)
Mean squared error (MSE)
+
If 𝑇 is an estimator of 𝜏(𝜃), then the MSE of 𝑇 is given by 𝑀𝑆𝐸(𝑇) = 𝐸6𝑇 − 𝜏(𝜃)7
+
Moreover, 𝑀𝑆𝐸(𝑇) = 𝑉(𝑇) + 6𝐵𝑖𝑎𝑠(𝑇)7 , so if 𝑇 is unbiased, the 𝑀𝑆𝐸(𝑇) = 𝑉(𝑇)
Uniformly minimum variance unbiased estimator (UMVUE)
An estimator 𝑇 ∗ of 𝜏(𝜃) is called a UMVUE of 𝜏(𝜃) is 𝑇 ∗ is unbiased for 𝜏(𝜃) and for any other
unbiased estimator 𝑇 of 𝜏(𝜃), 𝑉(𝑇 ∗ ) ≤ 𝑉(𝑇) for all 𝜃 ∈ Ω
Cramer-Rao lower bound (CRLB)
!
-." ())1
If 𝑇 is an unbiased estimator of 𝜏(𝜃), then the CRLB is 𝑉(𝑇) ≥ # ! or
"23 4567(8;)):;
#$
, !
-." ())1
𝑉(𝑇) ≥ #!
. If an unbiased estimator equals the CRLB, it is a UMVUE
<"2= ! 4567(8;)):>
#$
Relative efficiency
The relative efficiency of an unbiased estimator 𝑇 of 𝜏(𝜃) to another unbiased estimator 𝑇 ∗ of
?(@ ∗ )
𝜏(𝜃) is given by 𝑟𝑒(𝑇, 𝑇 ∗ ) = ?(@)
Efficiency
An unbiased estimator 𝑇 ∗ of 𝜏(𝜃) is said to be efficient if 𝑟𝑒(𝑇, 𝑇 ∗ ) ≤ 1 for all unbiased
estimators 𝑇 of 𝜏(𝜃), and all 𝜃 ∈ Ω. The efficiency of an unbiased estimator 𝑇 of 𝜏(𝜃) is given by
𝑒(𝑇) = 𝑟𝑒(𝑇, 𝑇 ∗ ) if 𝑇 ∗ is an efficient estimator of 𝜏(𝜃).
An unbiased estimator that reaches the CRLB is efficient and an efficient estimator is UMVUE
Estimator with several unknown parameters
If 𝑋! , … , 𝑋" have pdf 𝑓(𝑥; 𝜃! , … , 𝜃' ), solve the system of equations with 𝑘 equations.
! !
For MME, the system is: 𝐸(𝑋! ) = " ∑"#$! 𝑋#! , … , 𝐸(𝑋 ' ) = " ∑"#$! 𝑋#'
For MML, the system is all the 𝑘 partial derivatives of the log-likelihood equal to 0
Week 2
Simple consistency
Let {𝑇" } be a sequence of estimators of 𝜏(𝜃). These estimators are said to be consistent
estimators of 𝜏(𝜃) if for every 𝜀 > 0, lim 𝑃(|𝑇" − 𝜏(𝜃)| < 𝜀) = 1 for every 𝜃 ∈ Ω
"→B
C
This is equivalent to lim 𝑃(|𝑇" − 𝜏(𝜃)| > 𝜀) = 0. The notation is 𝑇" → 𝜏(𝜃)
"→B
A consistent estimator converges in probability to the true value
MSE consistency
If {𝑇" } be a sequence of estimators of 𝜏(𝜃), then they are called MSE consistent if
+
lim 𝐸6𝑇" − 𝜏(𝜃)7 = 0 for every 𝜃 ∈ Ω. MSE consistency ⟹ (simple) consistency
"→B
Asymptotically unbiased estimator
A sequence of estimators {𝑇" } is said to be asymptotically unbiased for 𝜏(𝜃) if lim 𝐸(𝑇" ) = 𝜏(𝜃)
"→B
for all 𝜃 ∈ Ω. Estimator is MSE consistent ⟺ it is asymptotically unbiased and lim 𝑉( 𝑇" ) = 0
"→B
Law of Large Numbers (LLN)
If 𝑋! , … , 𝑋" is a random sample from a distribution with finite mean and variance, then
! C
𝑋+" = " ∑"#$! 𝑋# → 𝐸(𝑋)
Continuous mapping theorem
C C
If 𝑌" → 𝑐, then for any function 𝑔(𝑦) that is continuous at 𝑐, 𝑔(𝑌" ) → 𝑔(𝑐)
This is also valid for 𝑘-dimensional vectors
Convergence in probability results theorem
C C
If 𝑋" and 𝑌" are two sequences of random variables such that 𝑋" → 𝑐 and 𝑌" → 𝑑, then:
C
- 𝑎𝑋" + 𝑏𝑌" → 𝑎𝑐 + 𝑏𝑑
C
- 𝑋" 𝑌" → 𝑐𝑑
C
- 𝑋" /𝑐 → 1 for 𝑐 ≠ 0
C
- 1/𝑋" → 1/𝑐 if 𝑐 ≠ 0
C
- g𝑋" → √𝑐 if 𝑐 > 0
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper LeonVerweij. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €6,99. Je zit daarna nergens aan vast.
Is Stuvia te vertrouwen?
4,6 sterren op Google & Trustpilot (+1000 reviews)
Afgelopen 30 dagen zijn er 53340 samenvattingen verkocht
Opgericht in 2010, al 14 jaar dé plek om samenvattingen te kopen