[Solved] Markov Decision Process example model?

These are a bit more elaborate than a simple example model, but are probably of interest since they are applied examples. They are Markov models of the COVID-19 pandemic, projecting hospitalizations, ICU needs, case counts and deaths under different mitigation strategies. The first is by Rob Brown. The second was by Jorge Muro-Arbulú, who adapted Rob's model to the country of Peru. I wrote a blog article about Rob's model last spring, and is in Spanish. Jorge put together an incredible brochure about the Peru model.

In this simple finite-state MDP, a taxi driver can be in one of three towns. In each town he can either cruise the streets, wait at a taxi stand, or wait for call from dispatch. The goal of the model is to compute the optimal policy (i.e., which action to do from each town). The town that he is in is the state.

I demonstrated value iteration with discounting (can be zero) on a finite horizon, and then also policy iteration on an infinite horizon using Iterate (discount rate must be greater than zero). I lifted the example from slides from Dennis Bricker at U. of Iowa.