Columbia Business School - First Year of the Doctoral Program in Decisions, Risk and Operations
• Stochastic processes
o Notes from Prof Assaf Zeevi's "Foundations of Stochastic Modelling".
o Notes from Prof David Yao's "Stochastic Processes II".
Conditional expectations
Measure theory
o In a probability space (W, , ) , a sigma field is a collection of events, each of
which as a subset of W . It satisfies (i) Æ Î (ii) A Î Ac Î (iii)
Ai Î Èi¥=1 Ai Î . Notes:
(i) and (ii) W Î
( )
c
¥ ¥
A =
i =1 i
Ac , so also closed under infinite (and finite) intersection.
i =1 i
o A random variable maps X (w) : W . When we say X is measurable with
respect to and write X Î , we mean {w : X (w) £ x } Î "x .
Conditional expectations
o (X | Y ) is a random variable. (X | Y )(w) = (X | Y = Y (w)) . In other words,
the fact Y = Y (w) “reveals” a “region” of W in which we are located. We then
find the expected value of X given we are in that “region”.
In terms of the definition below, we can write (X | Y ) = (X | s(Y )) ,
where s(Y ) is the sigma-field generated by Y – in other words,
s(Y ) = {{w : Y (w) £ x } : x Î } – every event can would be revealed by
Y.
o W = (X | ) is a random variable. (X | )(w) is a bit harder to understand –
effectively, it takes the expectation of X over the smallest that contains w . In
other words, let A be the smallest element of that contains w – then we
restrict ourselves to some region of W and find the expectation over that region;
(X | )(w) = (X A ) . Formal properties:
Daniel Guetta
,Stochastic Processes II Page 2
W Î : information as to where we are in W only ever “reaches” us via
knowledge of which part of we’re in, so this is obvious.
(W A ) = (X A ) for all A Î : we are now restricting ourselves to a
region of W that is -measurable. Provided A is the smallest element for
which w Î A , W (w) = (X A ) , and the result follows trivially. (If it is
not the smallest element, the result requires additional thought).
o Some properties
i. éêëX | ùúû if X Î
ii. éê éêëX | ùúû ùú = (X )
ë û
iii. (XZ | ) = Z (X | ) if Z Î
iv. Tower: éê (X | ) | ùú = (X | ) if Í : in this case, is “more
ë û
descriptive” than , so the result makes sense.
( ( ) )
Proof: Use (X | ) | A = (X | ) A ( ) for A Î . Then use
the fact that A Î to show this is equal to (X A ) .
v. Linearity
(
vi. Jensen’s: for convex f, éêë f (X ) | ùûú ³ f éêëX | ùúû )
o Notes
éêëX ùúû = éêëX | {Æ, W}ùúû (the RHS is a constant, because whatever w we
choose, the only element of {Æ, W} that contains it will be W ). Thus, (ii)
is a special case of (iv).
Integrability of X implies integrability of éêëX | ùûú :
(vi) (ii)
é
ë
ù
û ë (
ê (X | ) ú £ éê X | )ù = X
ûú ( )
o Example: Let W be countable. Let = {1 , 2 , } be a partition of W , and
å w Î i X ( w )( w )
be the set of all subsets of . Then (X | ) takes value ( i )
with
probability (i ) .
Daniel Guetta
, Stochastic Processes II Page 3
Proof: Clearly, the RV is measurable, because each value it can take is
defined by a i . Also, éêë (X | )A ùûú is the expected value over those i Í A .
Clearly, = éëêX A ùûú .
Martingales
Definition: {Xn } is a sub-martingale with respect to {n } (where n Î n+1 ) if
i. X n Î n
ii. (X n ) < ¥ [it is often convenient to work with the stronger condition
Xn < ¥ ].
iii. éëêXn +1 | n ùûú ³ Xn [< gives a super-martingale, = gives a martingale]. Implies the
weaker property éêëXn +1 ùûú ³ éëêXn ùûú
Remarks:
o A convex function of a martingale is a submartingale.
o An increasing convex function of a submartingale is a submartingale.
Proof: (i) and (ii) are simple. éêë f (Xn +1 ) | n ùúû ³ f ( [Xn +1 | n ]) ³ f (Xn ) .
å
n
Example: Let S n = i =1
X i , where the Xi are IID with (Xi ) = 0, Xi < ¥
o Sn is a martingale [ Sn £ n X1 ] (the mean martingale).
o If ar(X i ) = s 2 < ¥ , X n2 - s 2n is a martingale (the variance martingale).
qX1 qSn
Example (the exponential martingale): Let j(q) = (e ) . Mn = e / jn (q) is a
å
n
martingale. For example, if S n = i =1
Xi is an asymmetric random walk with
( )
Sn
1-p
p = (X i = 1) = 1 - (X i = -1) , then M n = p
is an exponential martingale, with
1-p
eq = p
and j(q) = 1 .
Example: Suppose an urn starts with one black and one white ball. We pull out balls
from the urn, and return them to the urn with another, new ball of the same color. Yn,
the proportion of white balls after n draws, is a martingale (mean ½).
Example: Let {Xn } be a Markov Chain with transition matrix P(x, y) and let h(x) be a
bounded function with h(x ) = å y p(x, y )h(y ) . {h(Xn )} is then a martingale.
Daniel Guetta
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller tandhiwahyono. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $2.64. You're not tied to anything after your purchase.
