100% de satisfacción garantizada Inmediatamente disponible después del pago Tanto en línea como en PDF No estas atado a nada
logo-home
Buscado previamente por ti
Buscado previamente por ti
logo
Markov Decision Processes Finals V2 10,78 €
Añadir al carrito
Navegar rápidamente a
Vista previa
  2.  
  3. Ma-rk-ov Decision Processes Fin V2
  4.  
  5. Ma-rk-ov Decision Processes Fin V2

Examen

Markov Decision Processes Finals V2

 0 veces vendidas
  • Grado
  • Ma-rk-ov Decision Processes Fin V2
  • Institución
  • Ma-rk-ov Decision Processes Fin V2

Markov Decision Processes Finals V2 A Markov Process is a process in which all states do not depend on previous actions. ️️True, Markov means that you don't have to condition on anything past the most recent state. A Markov Decision Process is a set of Markov Property Compliant states, wi...

[Mostrar más]

Vista previa 2 fuera de 14  páginas

Información del documento

  • 30 de octubre de 2024
  • 14
  • 2024/2025
  • Examen
  • Preguntas y respuestas

Temas

  • markov decision processes finals v2

Escuela, estudio y materia

  • Ma-rk-ov Decision Processes Fin V2
  • Ma-rk-ov Decision Processes Fin V2

Vendedor

avatar-seller
CertifiedGrades

Comentarios recibidos

Vista previa del contenido

Markov Decision Processes Finals V2

A Markov Process is a process in which all states do not depend on previous actions. ✔️✔️True,
Markov means that you don't have to condition on anything past the most recent state. A Markov
Decision Process is a set of Markov Property Compliant states, with rewards and values.



Decaying Reward encourages the agent to end the game quickly instead of running around and
gathering more reward ✔️✔️True, as reward decays the total reward for the episode decreases, so
the agent is encouraged to maximize total reward by ending the game quickly.



R(s) and R(s,a) are equivalent. ✔️✔️True, it just happens that it's easier to think about one vs the
other in certain situations.



Reinforcement Learning is harder to compute than a simple MDP. ✔️✔️True, you can just use the
Bellman Equations for an MDP, but Reinforcement Learning requires that you make observations and
then summarize those observations as values.



An optimal policy is the best possible sequence of actions for an MDP. ✔️✔️True, with a single caveat.
The optimal policy is a policy that maximizes reward over an entire episode by taking the argmax of
resulting values of actions + rewards. But MDPs are memoryless, so there is no concept of "sequence"
for a policy.



Temporal Difference Learning is the difference in reward you see on subsequent time steps.
✔️✔️False, Temporal Difference Learning is the difference in value estimates on subsequence time
steps.



RL falls generally into 3 different categories: Model-Based, Value-Based, and Policy-Based. ✔️✔️True,
Model-Based is essentially using the Bellman Equations to solve a problem, Value-Based is Temporal
Difference Learning, and Policy-Based is similar to Value-Based, but it solves in a finite amount of time
with a certain amount of confidence (in Greedy it's guaranteed).

, TD Learning is defined by Incremental Estimates that are Outcome Based. ✔️✔️True, TD Learning
thinks of learning in terms of "episodes", which it uses to estimate the transition functions rather than
having a predefined model.



For a learning rate to guarantee convergence, the sum of the learning rate must be infinite, and the sum
of the learning rate squared must be finite. ✔️✔️True, this is called a contraction mapping and it
guarantees convergence.



All of the TD learning methods have set backs, TD(1) is inefficient because it requires too much data and
has high variance, TD(0) has a maximum likelihood estimate but is hard to calculate for long episodes.
✔️✔️True, this is why we use TD(Lambda), which has many of the benefits of TD(0) but is much more
performant. Empirically, lambdas between 0.3 and 0.7 seem to perform best.



To control learning, you simply have the operator choose actions in addition to learning. ✔️✔️True,
states are experienced as observations during learning, so the operator can influence learning.



Q-Learning converges ✔️✔️True, the Bellman Equation satisfies a Contraction Mapping where the
sum of all is infinite, but the sum of all squared is less than infinite. It always converges to Q*.



As long as the update operators for Q-learning or Value-iteration are non-expansions, then they will
converge. ✔️✔️True, there are expansions that will converge, but only non-expansions are
guaranteed to converge independent of their starting values.



A convex combination will converge. ✔️✔️False, it must be a fixed convex combination to converge. If
the value can change, like with the Boltzmann exploration, then it is not guaranteed to converge.



In Greedy Policies, the difference between the true value and the current value of the policy is less than
some epsilon value for exploration. ✔️✔️True



It serves as a good check for how long we run value iteration until we're pretty confident that we have
the optimal policy. ✔️✔️True



For a set of linear equations, the solution can be found in polynomial time. ✔️✔️True

Los beneficios de comprar resúmenes en Stuvia estan en línea:

Garantiza la calidad de los comentarios

Garantiza la calidad de los comentarios

Compradores de Stuvia evaluaron más de 700.000 resúmenes. Así estas seguro que compras los mejores documentos!

Compra fácil y rápido

Compra fácil y rápido

Puedes pagar rápidamente y en una vez con iDeal, tarjeta de crédito o con tu crédito de Stuvia. Sin tener que hacerte miembro.

Enfócate en lo más importante

Enfócate en lo más importante

Tus compañeros escriben los resúmenes. Por eso tienes la seguridad que tienes un resumen actual y confiable. Así llegas a la conclusión rapidamente!

Preguntas frecuentes

What do I get when I buy this document?

You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.

100% de satisfacción garantizada: ¿Cómo funciona?

Nuestra garantía de satisfacción le asegura que siempre encontrará un documento de estudio a tu medida. Tu rellenas un formulario y nuestro equipo de atención al cliente se encarga del resto.

Who am I buying this summary from?

Stuvia is a marketplace, so you are not buying this document from us, but from seller CertifiedGrades. Stuvia facilitates payment to the seller.

Will I be stuck with a subscription?

No, you only buy this summary for 10,78 €. You're not tied to anything after your purchase.

Can Stuvia be trusted?

4.6 stars on Google & Trustpilot (+1000 reviews)

45,681 summaries were sold in the last 30 days

Founded in 2010, the go-to place to buy summaries for 15 years now

Empieza a vender
10,78 €
  • (0)
Añadir al carrito
Añadido