STOCHASTIC DYNAMIC PROGRAMMING

This concept came to be in the year 1957, thanks to Richard Bellman. He introduced this technique which gets used as a way to model decision making processes in situations of uncertainty. This type of programming uses a Bellman equation in a bid to show the problem that is to get solved.

The primary goal is to present options which can get used when making decisions when you are not sure of the future. Through coming up with possible outcomes to a solution, you get to decide on which one is most likely to occur as well as which option is the best under the circumstances.

An example in gambling

Did you know that you can make use of this concept when placing wagers? Suppose a gambler has two dollars which they intend to bet with and make some cash out of their effort. Let us imagine that the player can only play a game of chance a total number of four times and they intend to walk away having tripled the two dollars to six at the end of the game.

Let us say that the gambler starts off by betting $b on one of the plays in the game. Once they place that wager, they have a probability of 0.4 of winning, and if the player ends up doing so, will have added $b to what they have in hand. On the other hand, if they lose given the probability of 0.6 for losing, they will lose the wager by the amount they bet which is $b.

Keep in mind that all the plays are pairwise independent and as such, when the gambler makes a bet, they cannot place more than what they had when the game started. This stage is where stochastic dynamic programming comes in. You can make use of it to make a model of the problem so that you can decide on which betting approach will help you multiply your initial capital.

While using this technique, you can create a horizon from which you can choose the best outcome. However, if the number of plays is not limited like in the case above, it falls into the St. Petersburg Paradox.

Dynamic Programming for Partially Observable Stochastic Games

Stochastic games got introduced in the 1950s by Lloyd Shapley. This game is very dynamic, and it gets played by one or more people in stages. Every stage has a state at its beginning and players receive payoffs based on this condition as well as their actions in the game. After this, the game moves on to another state whose nature depends on the activities of the previous one.

Stochastic dynamic programming is usable in partially observable stochastic games to give a framework which enables players to plan under situations of uncertainty. There are many computations undertaken in the process. However, the solving process becomes difficult once you have gone through a few decision cycles.

Stochastic dynamic programming is excellent for use in various aspects of life whether you are gaming or deciding on the type of business in which to venture.