Overview on:
- Score Statistics, Maximum Likelihood Estimation (MLE), Fisher Information
- Sufficiency and Completeness with Neyman's Factorisation Theorem and Exponential Family
- Parameter Estimation: Desirable Properties of Estimators, Method of Moments, MLE and its properties
- Cramer-Rao L...
we want to infor the parameters ! In this case ,
we are parametric :
models
Chapter 1 :
a) Likelihood are functions of an unknown parameter .
O
110(U) = Px(NIO) < for a
single case
IID obs , due to independence
(1014) = Px(ip)
,
roan
↳ can
use (101) =
log((10ln)) .
Why ? log transformation
log is an
increasing
is one-to-one
,
function
, >N
Why MLE ? Likelihood
says how likely a value of the parameter is
given the data
to inter from the data :
maximise the likelihood
↳ find of the the data
mostly likely value parameter given .
b) Score Statistic , VIX
vix)
0 ETu(x)]
=
l'10m)
0
=
vologPx(n10) - derivative of
log-likelihood !
Fisher information :↑210) ; distribution changes
·
quickly
=
when
>
-
② var(v(x)) = -
Ele"(Olns] =
210) O departs from 00 ; Oo
estimate well
&
210) =
nicd
c) Sufficiency :
Def :
A partition A of sample space - is called sufficient for O if for all AjzA ,
Px/MO , MEAj)
is independent of 0
.
sufficient stat of without .
0
>
knowing a we can find the probablity an event the need to know
-
,
>
-
E at least I sufficient partition ,
ie.
knowing all the n Idatal outcome
↳ minimal Sufficient
Def If sufficient partition A sufficient paration B set
:
is
a such that
given any other , any
element of B is contained in a set element of A then A is said to be minimal sufficient
.
,
>
If T isAncient
for O and Amplete then T is minimal sufficient (Bahadur Incorem (
-
.
,
, TO FIND SUFFICIENT STATISTIC !!
① Meyman's Factorisation Theorem to this !
- My show
A statistic T is sufficient for 0 Px(u10) =
g(0 , T(x)) n(X) ·
② Exponential Family
If X, Xn are IID from a dist of the
exp family
·
·
,
.
...
Px(n10) =
expLACOBIn) + <10) + D(u)} =
try to show this
lif you start with I obe,
Then T = ZBIXi) is sufficient for 0 to because the [BIxi)
state
.
need
clearly
comes from likelihood :
TTPx (n/0)
from def .
>
-
⑤ T is sufficient for O If
(i) for all n and a such that TIu) = a
,
Px(n10 , T(n) =
a) is independent of .
0
(ii) for all 2 and I',
T(u) E PxIMIO) is of O
=
T(u'l
independent
↑x (n'10)
AND T is minimal sufficient if(you show the other direction ,
is ,
E)
Px(n(0) => T() =
TIu'l
↑ x In '10)
d) completeness
family [PX (410) 083
&
Def :
A :
of distributions on - is called complete
if E[hix)] = 0 for all 00 >
- P(nIX) =
010) = 1 for all .
OE h(X) is a zero function
any statistic n(x)
for such that the above expectation makes sense .
>
- A statistic T is said to be
complete if its
family of distributions &P + CtIO) 08] :
is complete
① Exponential
family
Suppose (X , Xn)
X is IID sample from the probability model
=
..., an
Px(n(0) =
expCAIOBIn) + col + DinI] ,
Do
and let T = [B(Xi) denote the corresponding sufficient statistic.
If & contains an open interval ,
then T is .
complete
↓ (R 1 - 0 , 3)
Eg :
,
space of O
, *
completeness of X =
completeness of the
family of distributions of .
X
Chapter 2 :
Goal :
estimate gloy ,
a function of 0
.
Setting : Let (x) be our estimate of g(0) when we observe X =
.
a
An estimator is a function of the r v
.
.
XI, ...,
Xn
.
-
L 1) Estimator not
should take values outside the parameter space .
I
2) Unbiasedness
E[(x)) =
g(0) +
bg(0)
want bg10) = 0 g(x)] =
g(0) VOzO
desirable 3) small volatility
of Squared Error IMSE) > 0
properties mean -
it .
for every 320 , PlIIX)-gl01K) < 10) + 0 as n+
>
-
E(g(X)] + g(0) and var(g(x)) + 0 as n + & = g(X) is consistent for glo)
construct estimators :
a) method of moments
for the r-th moment, E(x] + x
>
n+ 0
-
,
Exr] = Xi [ law of
large numbers]
[theoretical] =
[data]
Exi
Eg
: Eix] =
N
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller nghueyern. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $9.99. You're not tied to anything after your purchase.
