Discounting, Ergodicity and Convergence for Markov Decision Processes

Dynamic Programming

1957 • 13,549 citations

Dynamic Programming and Markov Processes.

1961 • 3,612 citations

Finite Markov chains

1976 • 2,965 citations

Finite Markov Chains.

1960 • 1,640 citations

Discrete Dynamic Programming

1962 • 822 citations

The Optimality of (S,s) Policies in the Dynamic Inventory Problem

2005 • 731 citations

Studies in the Mathematical Theory of Inventory and Production

1960 • 559 citations

Contraction Mappings in the Theory Underlying Dynamic Programming

1967 • 463 citations

Discrete Dynamic Programming with Sensitive Discount Optimality Criteria

1969 • 264 citations

Dynamic programming, Markov chains, and the method of successive approximations

1963 • 216 citations

Limits for the characteristic roots of a matrix. IV: Applications to stochastic matrices

1952 • 207 citations

A modified dynamic programming method for Markovian decision problems

1966 • 135 citations

The ergodic properties of non-homogeneous finite Markov chains

1956 • 87 citations

On the Iterative Method of Dynamic Programming on a Finite Space Discrete Time Markov Process

1965 • 69 citations

Turnpike Planning Horizons for a Markovian Decision Model

1968 • 62 citations

The asymptotic behaviour of the minimal total expected cost for the denumerable state Markov decision model

1975 • 26 citations

Existence and Uniqueness Theorems for the Optimal Inventory Equation

1966 • 8 citations

Turnpike Theory in Deterministic Discrete Dynamic Programming with Discount Factor Greater than One

1973 • 7 citations

On a Conjecture of Iglehart

1975 • 6 citations