DISCOUNTED AND UNDISCOUNTED VALUE-ITERATION IN MARKOV DECISION PROBLEMS: A SURVEY
Data up to Jan 2025
Total Citations Per Year
Abstract
References (44)
Dynamic Programming and Markov Processes.
1961 • 3,612 citations
A Markovian Decision Process
1957 • 2,197 citations
Nonlinear Programming: A Unified Approach.
1970 • 1,066 citations
Finite State Markovian Decision Processes
1970 • 877 citations
Discrete Dynamic Programming
1962 • 822 citations
Applying a New Device in the Optimization of Exponential Queuing Systems
1975 • 749 citations
Contraction Mappings in the Theory Underlying Dynamic Programming
1967 • 463 citations
Dynamic programming, Markov chains, and the method of successive approximations
1963 • 216 citations
Markov-Renewal Programming. I: Formulation, Finite Return Models
1963 • 186 citations
Iterative solution of the functional equations of undiscounted Markov renewal programming
1971 • 175 citations
Multichain Markov Renewal Programs
1968 • 160 citations
A modified dynamic programming method for Markovian decision problems
1966 • 135 citations
On Finding the Maximal Gain for Markov Decision Processes
1969 • 118 citations
Some Bounds for Discounted Sequential Decision Processes
1971 • 99 citations
Optimal decision procedures for finite markov chains. Part I: Examples
1973 • 84 citations
Computing Methods in Optimization Problems
1969 • 82 citations
Bounds and Transformations for Discounted Finite Markov Decision Chains
1975 • 77 citations
On the Iterative Method of Dynamic Programming on a Finite Space Discrete Time Markov Process
1965 • 69 citations
The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms
1978 • 63 citations
Exponential convergence of products of stochastic matrices
1977 • 63 citations
Turnpike Planning Horizons for a Markovian Decision Model
1968 • 62 citations
Discounting, Ergodicity and Convergence for Markov Decision Processes
1977 • 55 citations
A set of successive approximation methods for discounted Markovian decision problems
1976 • 54 citations
Generalization of White's Method of Successive Approximations to Periodic Markovian Decision Processes
1972 • 54 citations
The Asymptotic Behavior of Undiscounted Value Iteration in Markov Decision Problems
1977 • 49 citations
Geometric convergence of value-iteration in multichain Markov decision problems
1979 • 46 citations
Accelerated procedures for the solution of discrete Markov control problems
1971 • 42 citations
Solution of a Markovian decision problem by successive overrelaxation
1973 • 31 citations
Tests for Suboptimal Actions in Discounted Markov Programming
1973 • 31 citations
Technical Note—Accelerated Computation of the Expected Discounted Return in a Markov Chain
1978 • 30 citations
Technical Note—Bounds on the Gain of a Markov Decision Process
1971 • 29 citations
Note—A Test for Nonoptimal Actions in Undiscounted Finite Markov Decision Chains
1976 • 28 citations
The asymptotic behaviour of the minimal total expected cost for the denumerable state Markov decision model
1975 • 26 citations
Zur Extrapolation in Markoffschen Entscheidungsmodellen mit Diskontierung
1974 • 22 citations
Étude asymptotique des systèmes markoviens à commande
1967 • 22 citations
On Maximal Rewards and $|varepsilon$-Optimal Policies in Continuous Time Markov Decision Chains
1974 • 22 citations
Technical Note—Elimination of Suboptimal Actions in Markov Decision Problems
1973 • 21 citations
Multiple Policy Improvements in Undiscounted Markov Renewal Programming
1971 • 17 citations
Optimization of Discounted Markov Decision Problems
1969 • 16 citations
A Modified Form of the Iterative Method of Dynamic Programming
1975 • 14 citations
An iterative method for approximating average cost optimal (s,S) inventory policies
1974 • 6 citations
Preferred Rules in Continuous Time Markov Decision Processes
1974 • 3 citations
The functional equations of undiscounted markov renewal programming : (prepublication)
1976 • 2 citations
Deleted Work
1955 • 0 citations