Stochastic Processes II Page 1
STOCHASTIC PROCESSES II
PART I – MARTINGALES
Conditional expectations
Measure theory
o In a probability space (W, , ) , a sigma field is a collection of events, each of
which as a subset of W . It satisfies (i) Æ Î (ii) A Î Ac Î (iii)
Ai Î Èi¥=1 Ai Î . Notes:
(i) and (ii) W Î
( )
c
¥ ¥
A =
i =1 i
Ac , so also closed under infinite (and finite) intersection.
i =1 i
o A random variable maps X (w) : W . When we say X is measurable with
respect to and write X Î , we mean {w : X (w) £ x } Î "x .
Conditional expectations
o (X | Y ) is a random variable. (X | Y )(w) = (X | Y = Y (w)) . In other words,
the fact Y = Y (w) “reveals” a “region” of W in which we are located. We then
find the expected value of X given we are in that “region”.
In terms of the definition below, we can write (X | Y ) = (X | s(Y )) ,
where s(Y ) is the sigma-field generated by Y – in other words,
s(Y ) = {{w : Y (w) £ x } : x Î } – every event can would be revealed by
Y.
o W = (X | ) is a random variable. (X | )(w) is a bit harder to understand –
effectively, it takes the expectation of X over the smallest that contains w . In
other words, let A be the smallest element of that contains w – then we
restrict ourselves to some region of W and find the expectation over that region;
(X | )(w) = (X A ) . Formal properties:
Daniel Guetta
,Stochastic Processes II Page 2
W Î : information as to where we are in W only ever “reaches” us via
knowledge of which part of we’re in, so this is obvious.
(W A ) = (X A ) for all A Î : we are now restricting ourselves to a
region of W that is -measurable. Provided A is the smallest element for
which w Î A , W (w) = (X A ) , and the result follows trivially. (If it is
not the smallest element, the result requires additional thought).
o Some properties
i. éêëX | ùúû if X Î
ii. éê éêëX | ùúû ùú = (X )
ë û
iii. (XZ | ) = Z (X | ) if Z Î
iv. Tower: éê (X | ) | ùú = (X | ) if Í : in this case, is “more
ë û
descriptive” than , so the result makes sense.
( ( ) )
Proof: Use (X | ) | A = (X | ) A ( ) for A Î . Then use
the fact that A Î to show this is equal to (X A ) .
v. Linearity
(
vi. Jensen’s: for convex f, éêë f (X ) | ùûú ³ f éêëX | ùúû )
o Notes
éêëX ùúû = éêëX | {Æ, W}ùúû (the RHS is a constant, because whatever w we
choose, the only element of {Æ, W} that contains it will be W ). Thus, (ii)
is a special case of (iv).
Integrability of X implies integrability of éêëX | ùûú :
(vi) (ii)
é
ë
ù
û ë (
ê (X | ) ú £ éê X | )ù = X
ûú ( )
o Example: Let W be countable. Let = {1 , 2 , } be a partition of W , and
å w Î i X ( w )( w )
be the set of all subsets of . Then (X | ) takes value ( i )
with
probability (i ) .
Daniel Guetta
, Stochastic Processes II Page 3
Proof: Clearly, the RV is measurable, because each value it can take is
defined by a i . Also, éêë (X | )A ùûú is the expected value over those i Í A .
Clearly, = éëêX A ùûú .
Martingales
Definition: {Xn } is a sub-martingale with respect to {n } (where n Î n+1 ) if
i. X n Î n
ii. (X n ) < ¥ [it is often convenient to work with the stronger condition
Xn < ¥ ].
iii. éëêXn +1 | n ùûú ³ Xn [< gives a super-martingale, = gives a martingale]. Implies the
weaker property éêëXn +1 ùûú ³ éëêXn ùûú
Remarks:
o A convex function of a martingale is a submartingale.
o An increasing convex function of a submartingale is a submartingale.
Proof: (i) and (ii) are simple. éêë f (Xn +1 ) | n ùúû ³ f ( [Xn +1 | n ]) ³ f (Xn ) .
å
n
Example: Let S n = i =1
X i , where the Xi are IID with (Xi ) = 0, Xi < ¥
o Sn is a martingale [ Sn £ n X1 ] (the mean martingale).
o If ar(X i ) = s 2 < ¥ , X n2 - s 2n is a martingale (the variance martingale).
qX1 qSn
Example (the exponential martingale): Let j(q) = (e ) . Mn = e / jn (q) is a
å
n
martingale. For example, if S n = i =1
Xi is an asymmetric random walk with
( )
Sn
1-p
p = (X i = 1) = 1 - (X i = -1) , then M n = p
is an exponential martingale, with
1-p
eq = p
and j(q) = 1 .
Example: Suppose an urn starts with one black and one white ball. We pull out balls
from the urn, and return them to the urn with another, new ball of the same color. Yn,
the proportion of white balls after n draws, is a martingale (mean ½).
Example: Let {Xn } be a Markov Chain with transition matrix P(x, y) and let h(x) be a
bounded function with h(x ) = å y p(x, y )h(y ) . {h(Xn )} is then a martingale.
Daniel Guetta
STOCHASTIC PROCESSES II
PART I – MARTINGALES
Conditional expectations
Measure theory
o In a probability space (W, , ) , a sigma field is a collection of events, each of
which as a subset of W . It satisfies (i) Æ Î (ii) A Î Ac Î (iii)
Ai Î Èi¥=1 Ai Î . Notes:
(i) and (ii) W Î
( )
c
¥ ¥
A =
i =1 i
Ac , so also closed under infinite (and finite) intersection.
i =1 i
o A random variable maps X (w) : W . When we say X is measurable with
respect to and write X Î , we mean {w : X (w) £ x } Î "x .
Conditional expectations
o (X | Y ) is a random variable. (X | Y )(w) = (X | Y = Y (w)) . In other words,
the fact Y = Y (w) “reveals” a “region” of W in which we are located. We then
find the expected value of X given we are in that “region”.
In terms of the definition below, we can write (X | Y ) = (X | s(Y )) ,
where s(Y ) is the sigma-field generated by Y – in other words,
s(Y ) = {{w : Y (w) £ x } : x Î } – every event can would be revealed by
Y.
o W = (X | ) is a random variable. (X | )(w) is a bit harder to understand –
effectively, it takes the expectation of X over the smallest that contains w . In
other words, let A be the smallest element of that contains w – then we
restrict ourselves to some region of W and find the expectation over that region;
(X | )(w) = (X A ) . Formal properties:
Daniel Guetta
,Stochastic Processes II Page 2
W Î : information as to where we are in W only ever “reaches” us via
knowledge of which part of we’re in, so this is obvious.
(W A ) = (X A ) for all A Î : we are now restricting ourselves to a
region of W that is -measurable. Provided A is the smallest element for
which w Î A , W (w) = (X A ) , and the result follows trivially. (If it is
not the smallest element, the result requires additional thought).
o Some properties
i. éêëX | ùúû if X Î
ii. éê éêëX | ùúû ùú = (X )
ë û
iii. (XZ | ) = Z (X | ) if Z Î
iv. Tower: éê (X | ) | ùú = (X | ) if Í : in this case, is “more
ë û
descriptive” than , so the result makes sense.
( ( ) )
Proof: Use (X | ) | A = (X | ) A ( ) for A Î . Then use
the fact that A Î to show this is equal to (X A ) .
v. Linearity
(
vi. Jensen’s: for convex f, éêë f (X ) | ùûú ³ f éêëX | ùúû )
o Notes
éêëX ùúû = éêëX | {Æ, W}ùúû (the RHS is a constant, because whatever w we
choose, the only element of {Æ, W} that contains it will be W ). Thus, (ii)
is a special case of (iv).
Integrability of X implies integrability of éêëX | ùûú :
(vi) (ii)
é
ë
ù
û ë (
ê (X | ) ú £ éê X | )ù = X
ûú ( )
o Example: Let W be countable. Let = {1 , 2 , } be a partition of W , and
å w Î i X ( w )( w )
be the set of all subsets of . Then (X | ) takes value ( i )
with
probability (i ) .
Daniel Guetta
, Stochastic Processes II Page 3
Proof: Clearly, the RV is measurable, because each value it can take is
defined by a i . Also, éêë (X | )A ùûú is the expected value over those i Í A .
Clearly, = éëêX A ùûú .
Martingales
Definition: {Xn } is a sub-martingale with respect to {n } (where n Î n+1 ) if
i. X n Î n
ii. (X n ) < ¥ [it is often convenient to work with the stronger condition
Xn < ¥ ].
iii. éëêXn +1 | n ùûú ³ Xn [< gives a super-martingale, = gives a martingale]. Implies the
weaker property éêëXn +1 ùûú ³ éëêXn ùûú
Remarks:
o A convex function of a martingale is a submartingale.
o An increasing convex function of a submartingale is a submartingale.
Proof: (i) and (ii) are simple. éêë f (Xn +1 ) | n ùúû ³ f ( [Xn +1 | n ]) ³ f (Xn ) .
å
n
Example: Let S n = i =1
X i , where the Xi are IID with (Xi ) = 0, Xi < ¥
o Sn is a martingale [ Sn £ n X1 ] (the mean martingale).
o If ar(X i ) = s 2 < ¥ , X n2 - s 2n is a martingale (the variance martingale).
qX1 qSn
Example (the exponential martingale): Let j(q) = (e ) . Mn = e / jn (q) is a
å
n
martingale. For example, if S n = i =1
Xi is an asymmetric random walk with
( )
Sn
1-p
p = (X i = 1) = 1 - (X i = -1) , then M n = p
is an exponential martingale, with
1-p
eq = p
and j(q) = 1 .
Example: Suppose an urn starts with one black and one white ball. We pull out balls
from the urn, and return them to the urn with another, new ball of the same color. Yn,
the proportion of white balls after n draws, is a martingale (mean ½).
Example: Let {Xn } be a Markov Chain with transition matrix P(x, y) and let h(x) be a
bounded function with h(x ) = å y p(x, y )h(y ) . {h(Xn )} is then a martingale.
Daniel Guetta
