100% satisfaction guarantee Immediately available after payment Both online and in PDF No strings attached
logo-home
Previously searched by you
Previously searched by you
logo
Complete samenvatting Operations Research II (OR2) $3.75   Add to cart
Quickly navigate to
Preview
  2.  
  3. Vrije Universiteit Amsterdam (VU)
  4.  
  5. Operations Research II (E_EOR2_OR2)

Summary

Complete samenvatting Operations Research II (OR2)
 41 views  2 purchases
  • Course
  • Operations Research II (E_EOR2_OR2)
  • Institution
  • Vrije Universiteit Amsterdam (VU)
  • Book
  • Operations Research

A complete summary of the second-year econometrics course in operations research II. Every week, the summary reviews the material based on theory and examples.

Preview 3 out of 30  pages

Document information

  • Yes
  • March 9, 2023
  • 30
  • 2021/2022
  • Summary

Subjects

  • markov chains
  • markov
  • chains
  • stochastic process
  • transition matrix
  • transition probability
  • modeling with states
  • state space
  • transition probabilities
  • observation epochs
  • e state space
  • meaning of state
  • discrete time markov chains
  • one step transition probabili

Connected book

book image

Book Title:

Author(s):

  • Edition:
  • ISBN:
  • Edition:

More summaries for

  • Introduction to Operations Research summary
All for this textbook (2)

Written for

  • Vrije Universiteit Amsterdam (VU)
  • Econometrie en Operationele research
  • Operations Research II (E_EOR2_OR2)
All documents for this subject (1)

Seller

avatar-seller
victorvanderwel

Reviews received

Content preview

Summary operations research
II part I

,Contents

1 Discrete-time Markov Chains 3
1.1 Modeling with states, state space, transition probabilities . . . 3
1.2 Computing n-step transition probabilities; chapman-kolmogorov 4
1.3 Concepts of the state classification and statespace . . . . . . . 4
1.4 Limit theorems for ergodic Markov chains; equilibrium (sta-
tionary, steady-state) distribution; equilibrium equations . . . 5
1.5 First-passage time probabilities . . . . . . . . . . . . . . . . . 6
1.6 Absorbing Markov chains . . . . . . . . . . . . . . . . . . . . . 6

2 Continuous-time Markov Chains 7
2.1 Modelling with states, state space, transition rates . . . . . . . 7
2.2 Interpretation of transition rates and their construction . . . . 8

3 Poisson processes 10
3.1 Merging and Splitting . . . . . . . . . . . . . . . . . . . . . . 10

4 Queueing systems 11
4.1 balance equations . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 steady-state performance measures and how to compute these 11
4.3 effective arrival rate; Little . . . . . . . . . . . . . . . . . . . . 12
4.4 Poisson-exponential queues . . . . . . . . . . . . . . . . . . . . 12

5 Standard exam questions 14
5.1 Discret-time Markov chains . . . . . . . . . . . . . . . . . . . 14
5.2 Continuous-time Markov Chains . . . . . . . . . . . . . . . . . 14
5.3 Poisson process . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.4 Queueing theory . . . . . . . . . . . . . . . . . . . . . . . . . 18

6 Dynamic programming 21
6.1 Deterministic dynamic programming . . . . . . . . . . . . . . 21
6.2 Stochastic dynamic programming . . . . . . . . . . . . . . . . 22

7 Markov Decision Process 24
7.1 Infinite horizon . . . . . . . . . . . . . . . . . . . . . . . . . . 24
7.2 Policy iteration for the discounted cost criterion . . . . . . . . 25
7.2.1 Policy iteration for the long-run average cost criterion . 25
7.3 Value iteration for discounted criterion . . . . . . . . . . . . . 26
7.3.1 Stopping criterion . . . . . . . . . . . . . . . . . . . . . 26
7.3.2 Value iteration for the long-run average criterion . . . . 26

1

, 8 Q-Learning for the total discounted criterion 28
8.1 Ingredients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
8.2 Q-learning algorithm . . . . . . . . . . . . . . . . . . . . . . . 29




2

The benefits of buying summaries with Stuvia:

Guaranteed quality through customer reviews

Guaranteed quality through customer reviews

Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.

Quick and easy check-out

Quick and easy check-out

You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.

Focus on what matters

Focus on what matters

Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!

Frequently asked questions

What do I get when I buy this document?

You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.

Satisfaction guarantee: how does it work?

Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.

Who am I buying these notes from?

Stuvia is a marketplace, so you are not buying this document from us, but from seller victorvanderwel. Stuvia facilitates payment to the seller.

Will I be stuck with a subscription?

No, you only buy these notes for $3.75. You're not tied to anything after your purchase.

Can Stuvia be trusted?

4.6 stars on Google & Trustpilot (+1000 reviews)

67474 documents were sold in the last 30 days

Founded in 2010, the go-to place to buy study notes for 14 years now

Start selling
$3.75  2x  sold
  • (0)
  Add to cart