Skip to content
Notifications
Clear all

[Solved] Markov Decision Process example model?

3 Posts
3 Users
1 Likes
2,170 Views
Avatar
Posts: 4
Customer
Topic starter
(@qsw1wpg6oem829t8ebrrh7tlyok2)
Active Member
Joined: 3 years ago

Does anyone have a simple example model of a Markov Decision Process (MDP)?

2 Replies
Lonnie Chrisman
Posts: 39
Admin
(@lchrisman)
Member
Joined: 13 years ago

These are a bit more elaborate than a simple example model, but are probably of interest since they are applied examples. They are Markov models of the COVID-19 pandemic, projecting hospitalizations, ICU needs, case counts and deaths under different mitigation strategies. The first is by Rob Brown. The second was by Jorge Muro-Arbulú, who adapted Rob's model to the country of Peru. I wrote a blog article about Rob's model last spring, and is in Spanish. Jorge put together an incredible brochure about the Peru model.

Reply
Avatar
Posts: 25
Moderator
(@drice)
Member
Joined: 3 years ago

 

In this simple finite-state MDP, a taxi driver can be in one of three towns. In each town he can either cruise the streets, wait at a taxi stand, or wait for call from dispatch. The goal of the model is to compute the optimal policy (i.e., which action to do from each town). The town that he is in is the state.

I demonstrated value iteration with discounting (can be zero) on a finite horizon, and then also policy iteration on an infinite horizon using Iterate (discount rate must be greater than zero). I lifted the example from slides from Dennis Bricker at U. of Iowa. 

Reply
Share: