Dynamic Probabilistic Decision Processes

ABSTRACT 
This paper presents a general model for the formulation and solution of the risk-sensitive dynamic decision problem that maximizes the certain equivalent of the discounted rewards of a time-varying Markov decision process. The problem is solved by applying the principle of optimality and stochastic dynamic programming to the immediate rewards and the certain equivalent associated with the remaining transitions of a time-varying Markov process over a finite or infinite time horizon, under the assumptions of constant risk aversion and discounting of future cash flows. The solution provides transient and stationary optimal decision policies which depend on the presence or absence of discounting. The construction equipment replacement problem serves as an example application of the model to illustrate the solution methodology, and the sensitivity of the optimal policy to the discount factor and the degree of risk aversion.

AUTHORS 
Antonio Dias, Jr. and Photios G. Ioannou
Civil & Environmental Engineering Department
University of Michigan
Ann Arbor, Michigan 48109-2125, U.S.A.
e-mail: photios@umich.edu


KEYWORDS 

Dynamic programming, optimal policies, Markov process, decision analysis, uncertainty, discounting, risk aversion, engineering economy, equipment replacement.


AVAILABILITY 
Ioannou, P.G., "Dynamic Probabilistic Decision Processes," Journal of Construction Engineering and Management, ASCE, Vol. 115, No. 2, 237-257, June 1989.