Lecture notes MATH362 applied probability

MATH362

1 – AUXILIARY REVIEW OF PROBABILITY THEORY

1.1 probability space and distribution
Probability space – a triple ( Ω, F , P)
Sample space – Ω
- A non-empty set that describes all possible outcomes of the random
phenomenon
Events - F
- Set of all events which are suitable subsets of Ω (σ -algebra)
Probability distribution - P : F →[ 0,1]
Statements satisfied by probability distributions –
( i ) P ( Ω ) =1 (the probability of something happening is one
( ii ) for a sequence of pairwise disjoint events ( Ai ) ∞i=1 :
P ¿ (countable additivity property
Pairwise disjoint events - ( Ai ∩ A j=∅ , for i≠ j )
PROBABILITY DISTRIBUTIONS ON COUNTABLE SAMPLE SPACES:
1
Uniform probability distribution – let Ω be a finite set, P ( ω ) =
¿ Ω∨¿ ∀ ω ∈ Ω¿
Bernoulli distribution - P ( ω=1 )= p , P ( ω=0 )=1− p


{( )
n p k ( 1− p )n−k , if 0 ≤ k ≤ n
Binomial distribution - P ( ω=k ) = k
0 , otherwise
Multinomial distribution – given n ∈ Z +¿¿ and k ∈ Z +¿¿, let
Ω={( ω 1 , … , ω k ) : ωi ∈ { 0,1,2 , … , n } and ω 1+ …+ω k =n }
Then, for p1 , … , pk ≥ satisfying p1 +…+ pk =1, the multinomial distribution is defined:
n!
P ( ω1 , … , ωk ) = 1
p w … pwk
k

ω1 !… ωk ! 1

Geometric distribution - P ( ω=k ) =
ω −λ
{
p ( 1− p )k −1 , k ≥1
0 , ow
λ e
Poisson distribution - P λ ( ω )= ,ω∈N
ω!
CONTINUOUS PROBABILITY DISTRIBUTIONS:
❑
1
Uniform distribution - P ( A ) =
b−a A
∫ dx
❑

Exponential distribution - P ( A )=∫ λ e
−λx
dx
A
2
❑ − ( x−μ )

Gaussian distribution - P ( A )=∫ e 2σ
2
dx
A
❑
√ 2 π σ2
1
Gamma distribution - P ( A )= ∫
Γ ( z) A
z z−1 −λx
λ x e dx
❑

- Γ ( z ) : gamma function: Γ ( z )=∫ λ x e dx
z z−1 −λz

A

, 1.2 independent events
Independent events – two events A , B ⊆ Ω are independent ( A ⊥ B ) if
P ( A ∩B )=P ( A ) P(B)


Mutually independent events – events A1 , A 2 ,… , An ⊆Ω are mutually independent if
∀ I ⊆ { 1 , … ,n } :
P ( ¿ i∈ I A i )=∏ P ( A i )
i∈ I
Pairwise independent events - ∀ i, j ∈ I ,i ≠ j ,
P ( A i ∩ A j ) =P ( A i ) P (A j)
- Mutual independence ⇒ pairwise independence but not vice versa


1.3 conditional probability
Conditional probability – for two events A , B ⊆ Ω, P ( B )> 0, conditional probability of A
given B:
P( A ∩ B)
P ( A|B )=
P (B)
n
Law of total probability - P ( A )=∑ P ( A|Bi ) P( Bi )
i=1


1.4 random variable
Random variable – given a probability space ( Ω , F , P ) and a set S, a suitably nice function:
X :Ω → S is a S-valued random variable on ( Ω, F , P)
Distribution of a random variable – the distribution of X for suitable S:
P X ( A ) =P( { ω ∈ Ω : X ( ω ) ∈ A })
Distribution function of a real-valued random variable – the function F X :R → [0,1] given
by F X ( x )=P ( { ω ∈ Ω: X ( ω ) ≤ x } ) =: P ( X ≤ x ) , ∀ x ∈ R
Properties of distribution functions –
( i ) lim F ( x )=0 , lim F ( x ) =1
x →−∞ x →+∞
( ii ) x< y ⇒ F ( x ) ≤ F ( y )
( iii ) F is ¿ continuous i . e . F ( x+ h ) → F ( x ) as h → 0
Probability mass function of a discrete random variable – the distribution of X is given as a
function P X : S → [ 0,1 ] s .t .
P X ( x )=P ( { ω ∈Ω : X ( ω )=x } ) , ∀ x ∈ S
x
Distribution of a continuous random variable – F X ( x )=∫ f X ( y ) dy , x ∈ R
−∞
Probability density function of a continuous random variable – the integrable function
f X : R →¿
Independence of random variables – let ( Ω, F , P) be a probability space and
X 1 :Ω → S1 , … , X n : Ω→ S n be random variables on it
They are (mutually) independent if for suitable subsets: A1 ⊆ S 1 , … An ⊆S n :
n
P ( { ω : X 1 ( ω ) ∈ A1 , … , X n ( ω ) ∈ A n })=∏ P( {ω : X k ( ω ) ∈ A k })
k=1